27,417.3 tok/s, counting generated tokens · MLPerf Inference v5.0 · server
measured high confidenceServer result; benchmark harness and latency constraints apply.
Measured observations stay separate from modeled profiles. Follow the source registry for context.
27,417.3 tok/s, counting generated tokens · MLPerf Inference v5.0 · server
measured high confidenceServer result; benchmark harness and latency constraints apply.
30,724.5 tok/s, counting generated tokens · MLPerf Inference v5.0 · server
measured high confidenceServer result; benchmark harness and latency constraints apply.
92,338.7 tok/s, counting generated tokens · MLPerf Inference v5.0 preview · server
measured medium confidencePreview result; benchmark harness and latency constraints apply.
57,177 tok/s, counting generated tokens · MLPerf Inference v4.1 · server
measured high confidence50,796 tok/s, counting generated tokens · MLPerf Inference v4.1 · server
measured high confidence22,021 tok/s, counting generated tokens · MLPerf Inference v4.1 (ROCm) · server
measured high confidenceMLPerf Inference v4.1 closed-division Server result, quoted on AMD's page: AMD's own 8×MI300X submission with EPYC Turin logs 22,020.92 completed tokens per second, against NVIDIA's DGX H100 at 21,605.79. CPU, runtime and latency caveats apply.
21,605 tok/s, counting generated tokens · MLPerf Inference v4.1 (TensorRT) · server
measured high confidenceNVIDIA's own MLPerf Inference v4.1 closed-division Server result for DGX H100, 21,605.79 completed tokens per second in its submission log, which AMD's page quotes as the comparison for its MI300X row. [CORRECTED 2026-09-13: this note called the figure AMD's report, made without independent audit; it is NVIDIA's own submission, which AMD only quotes.]
24,110 tok/s, counting generated tokens · MLPerf Inference v4.1 (ROCm) · offline
measured high confidenceAMD's own MLPerf Inference v4.1 closed-division Offline result for 8×MI300X with EPYC Turin, 24,109.8 tokens per second in its submission log, against NVIDIA's DGX H100 at 24,524.9. [CORRECTED 2026-09-13: this row was rocm-mi300x-llama31-405b, labelled Llama 3.1 405B and linked to that model, because AMD's page labels the table row that way. Round 4.1 had no Llama 3.1 405B benchmark, and both figures are the Llama 2 70B Offline results in the two submissions' own logs. On the throughput page the row sat beside an audited 405B result some fifteen times lower per accelerator.]
24,525 tok/s, counting generated tokens · MLPerf Inference v4.1 (TensorRT) · offline
measured high confidenceNVIDIA's own MLPerf Inference v4.1 closed-division Offline result for DGX H100, 24,524.9 tokens per second in its submission log, quoted on AMD's page as the comparison. [CORRECTED 2026-09-13: this row was rocm-h100-llama31-405b, labelled Llama 3.1 405B and linked to that model, from the same mislabelled table row as its MI300X sibling, and its note called the figure AMD's report.]
69,251.4 tok/s, counting generated tokens · MLPerf Inference v6.0 · offline
measured high confidence8-GPU node; ~8,656 tok/s per GPU. Submission precision not independently confirmed.
58,553.3 tok/s, counting generated tokens · MLPerf Inference v6.0 · server
measured high confidence~7,319 tok/s per GPU; MLPerf latency-constrained server scenario.
673,936 tok/s, counting generated tokens · MLPerf Inference v6.0 · offline
measured high confidenceNebius submission; ~9,360 tok/s per GPU. CoreWeave submitted the same hardware class separately without publishing exact figures.
575,580 tok/s, counting generated tokens · MLPerf Inference v6.0 · server
measured high confidence~7,994 tok/s per GPU.
1,046,150 tok/s, counting generated tokens · MLPerf Inference v6.0 · offline
measured high confidence~14,530 tok/s per GPU; identical figures in NVIDIA's and Nebius's recaps.
1,096,770 tok/s, counting generated tokens · MLPerf Inference v6.0 · server
measured high confidence~15,233 tok/s per GPU.
468,665 tok/s, counting generated tokens · MLPerf Inference v6.0 · offline
measured high confidenceScale mismatch flagged: 16 nodes/64 GPUs, not a fully populated 72-GPU rack; ~7,323 tok/s per GPU.
1,183,326.9 tok/s, counting generated tokens · MLPerf Inference v6.1 preview · offline
measured medium confidenceNVIDIA's closed-division result in the preview category: 72 Vera Rubin GPUs of 288 GB HBM4 each, in 18 nodes of four, which is the whole linked rack. The weights are NVFP4, quantized from the original Hugging Face weights with no retraining (the checkpoint is named DeepSeek-R1-NVFP4-v2-mlpinf), and the rate belongs to that build. Run through the v6.1 endpoints harness, not LoadGen: the rate is the output tokens the harness counted over the whole run. The results table's inferred flag is 1 on this row and does not mean the run was converted from another scenario; the MLPerf Inference v6.1 source's notes say what it does mean. MLCommons calls a system available only once it is priced, has shipped to a third party and can be bought by further third parties without early-access approval; this one was entered as a preview system.
1,175,890.2 tok/s, counting generated tokens · MLPerf Inference v6.1 preview · server
measured medium confidenceThe Offline row's system, weights and harness. Server sends requests as a Poisson process at a target of 325 per second, and the rate is the output tokens over the window from the first request issued to the completion of the last one issued, not over the whole run. The checker holds DeepSeek-R1's Server run to a 99th-percentile time to first token of 2000 ms and time per output token of 80 ms. The results table's inferred flag is 1 here as well, with the same meaning as on the Offline row.
652,750.1 tok/s, counting generated tokens · MLPerf Inference v6.1 preview · interactive
measured medium confidenceThe Offline row's system, weights and harness, with the Interactive scenario's tighter latency limits: the checker holds DeepSeek-R1 to a 99th-percentile time to first token of 1500 ms and time per output token of 15 ms. Requests arrive as a Poisson process at a target of 170 per second, and the rate is the output tokens over the window from the first request issued to the completion of the last one issued. The results table's inferred flag is 1, with the same meaning as on the Offline row.
558,802.8 tok/s, counting generated tokens · MLPerf Inference v6.1 preview · offline
measured medium confidenceNebius's closed-division result in the preview category. Scale mismatch flagged: 9 nodes/36 GPUs of 288 GB HBM4 each, not a fully populated 72-GPU rack. The weights are NVFP4, quantized from the original Hugging Face weights with no retraining (the submission lists quantization and affine fusion; the checkpoint is named DeepSeek-R1-NVFP4-v2-mlpinf), and the rate belongs to that build. Run through the v6.1 endpoints harness, not LoadGen: the rate is the output tokens the harness counted over the whole run. The results table's inferred flag is 1 on this row and does not mean the run was converted from another scenario; the MLPerf Inference v6.1 source's notes say what it does mean. MLCommons calls a system available only once it is priced, has shipped to a third party and can be bought by further third parties without early-access approval; this one was entered as a preview system.
591,368.3 tok/s, counting generated tokens · MLPerf Inference v6.1 preview · server
measured medium confidenceThe Offline row's system, weights and harness; scale mismatch flagged there applies here. Server sends requests as a Poisson process at a target of 163 per second, and the rate is the output tokens over the window from the first request issued to the completion of the last one issued, not over the whole run. The checker holds DeepSeek-R1's Server run to a 99th-percentile time to first token of 2000 ms and time per output token of 80 ms. The results table's inferred flag is 1 here as well, with the same meaning as on the Offline row.
103,480 tok/s, counting generated tokens · MLPerf Inference v6.0 (ROCm, FP4) · offline
measured high confidence~12,935 tok/s per GPU; AMD states FP4 precision.
73,608 tok/s, counting generated tokens · MLPerf Inference v6.0 (ROCm, FP4) · interactive
measured high confidence~9,201 tok/s per GPU; v6.0's Interactive scenario applies a tighter per-token latency SLA.
27,804 tok/s, counting generated tokens · MLPerf Inference v5.1 · offline
measured high confidenceNeutral-ledger complement to the AMD-blog row (22,021 tok/s) — different harness and latency methodology; kept separate, never averaged.
8,840 tok/s, counting generated tokens · MLPerf Inference v5.1 · interactive
measured high confidence~1,105 tok/s per GPU.
68,781 tok/s, counting generated tokens · MLPerf Inference v5.1 · offline
measured high confidenceA second 8×MI325X submission (ASUSTeK) reported 68,121 offline — close but distinct; both valid, not averaged.
1,650 tok/s, counting generated tokens · MLPerf Inference v5.1 · offline
measured high confidence~206 tok/s per GPU; audited non-preview complement to the v5.0 preview Llama-2-70B row on the same hardware class.
3,225.9 tok/s per accelerator, counting tokens this repository has not identified · InferenceX dashboard (FP8 default) · 69 tok/s/user interactivity point
estimated medium confidencetok/s per GPU, interpolated between measured InferenceX runs (per the site's own methodology note); GPU count/TP not shown on the compare view. Which tokens it counts was not recorded: InferenceX derives three throughputs per GPU for every run, tput_per_gpu, input_tput_per_gpu and output_tput_per_gpu, and this row does not say which of them it was read from.
485.3 tok/s per accelerator, counting tokens this repository has not identified · InferenceX dashboard (FP8 default) · 69 tok/s/user interactivity point
estimated medium confidenceSame compare view as the B200 row; the ~6.6x gap is one interactivity slice, not a universal multiplier. Which of InferenceX's three per-GPU throughputs it is, total, input or output, was not recorded either.
2,816 tok/s per accelerator, counting generated tokens · vLLM v0.14.1, CUDA 13.0 · mixed-context (ISL 2K/OSL 1K), TP2
measured medium confidencetok/s per GPU on a 2-GPU dev node — deliberately NOT linked to the 72-GPU rack entity. Prefill-only on the same node: 7,360 tok/s per GPU. The post calls the 2,816 figure output throughput, so it counts generated tokens.
111 tok/s per user while decoding · vLLM day-0 build, FP8 KV cache · single user (batch 1), 8K in / 1K out
vendor reported medium confidencePer-user decode speed at batch 1, not aggregate serving throughput — the two differ by orders of magnitude and must not be compared directly. With DSpark speculative decoding the same configuration reaches 331 tokens/s per user, a 3.14x speed-up. Linked to the GB300 NVL72 with the 8 GPUs the run used. [CORRECTED 2026-09-13: this note said the row was not linked because the catalog has no 8-GPU GB300 system. A run on part of a rack links the rack and records the GPUs it used, as the SGLang rows beside it do.]
118 tok/s per user while decoding · vLLM day-0 build, FP8 KV cache · single user (batch 1), 8K in / 1K out
vendor reported medium confidencePer-user decode speed at batch 1 across 16 accelerators; 370 tokens/s with DSpark speculative decoding. Doubling tensor parallelism from TP8 buys only 6% single-user speed, which is the point: latency scales poorly with GPU count while aggregate throughput scales well. Linked to the GB300 NVL72 with the 16 GPUs the run used.
2,808 tok/s per accelerator, counting prompt and generated tokens · SGLang + Miles, prefill/decode disaggregated, fp4 arm · serving frontier at its throughput end, 8K in / 1K out, concurrency 1024
vendor reported medium confidenceThe throughput end of the serving frontier in SGLang's Kimi K3 post: 2,808 tok/s per GPU at 18.7 tok/s per user, with a client concurrency limit of 1,024 printed beside the point. The figure counts prompt and generated tokens and divides by all 16 GPUs of the arm, the prefill worker's included, so the run served about 44,928 tok/s, about 4,992 of them generated; at 18.7 tok/s per user that is about 267 streams decoding at once, well inside the 1,024 limit. The post does not state that denominator; the post's source record sets out the measurements it is inferred from. The arm is tagged fp4, which the post never defines, and the figure says its points come from a separate sweep; the untagged PP8 → TP8 arm peaks at 2,715. That sweep's concurrency-1 point decodes 115.8 tok/s per user, beside the ~113 tok/s the post gives for batch 1 before speculation and far from the ~423 it gives with DSpark, so the sweep most likely ran without speculation; the post does not say. [CORRECTED 2026-09-13: this row was sglang-k3-gb300-8gpu-batched and stored 22,464 tok/s, 2,808 decode tokens/s per GPU times 8 accelerators, left unlinked as an 8-GPU topology. The figure counts prompt tokens as well, and the run used 16 GPUs.]
2,715 tok/s per accelerator, counting prompt and generated tokens · SGLang + Miles, prefill/decode disaggregated · serving frontier, untagged PP8 → TP8 arm at its peak, 8K in / 1K out
vendor reported medium confidenceThe PP8 → TP8 · 1P:1D arm of the same frontier at its highest point, 2,715.1 tok/s per GPU at 18.74 tok/s per user, read from the SVG's data coordinates, whose axis ticks fit a straight line exactly; the figure prints no concurrency for it. It is the fp4 arm's topology without the fp4 tag. 16 GPUs × 2,715 = 43,440 tok/s counting prompt and generated tokens, about 4,827 of them generated, or about 258 streams at 18.74 tok/s. The per-GPU figure is divided by all 16 GPUs, the prefill worker's included: the post does not say so, and its source record infers it from the post's own measurements. The post names no precision and no speculation setting for its untagged arms. The caption calls arms within about 2% of one another indistinguishable.
2,633 tok/s per accelerator, counting prompt and generated tokens · SGLang + Miles, prefill/decode disaggregated, TP8 + DCP8 decode · serving frontier, the DCP composition at its peak, 8K in / 1K out
vendor reported medium confidenceThe caption's DCP composition, two PP8 prefill workers feeding two DCP8 decode nodes, at 2,633 tok/s per GPU; the figure plots it at 2,632.7 and 18.44 tok/s per user and prints no concurrency. Decode context parallelism shards the MLA cache inside each TP8 group, so a DCP8 decode node is still 8 GPUs and the arm is 32. 32 GPUs × 2,633 = 84,256 tok/s counting prompt and generated tokens, about 9,362 of them generated, or about 508 streams at 18.44 tok/s. The per-GPU figure is divided by all 32 GPUs, both prefill workers included: the post does not say so, and its source record infers it from the post's own measurements. The post names no precision and no speculation setting for its untagged arms. A single PP8 → DCP8 · 1P:1D arm peaks at 2,682.7 on 16 GPUs, within the 2% the caption calls indistinguishable.
541 tok/s, counting tokens this repository has not identified · SGLang + Miles, DCP8, hicache ratio 2 · AgentX replay of coding-agent sessions, 48 concurrent sessions
vendor reported low confidenceThe post's caption gives 541 tok/s at 48 sessions without saying what it counts, and only the alt text of the figure beside it calls that rate aggregate decode throughput. The row files it as unspecified, because the figure itself fits neither reading, as below. The run replays real coding-agent sessions on 2×4 GB300, 8 GPUs, with a host-memory KV tier at ratio 2 on both arms and decode context parallelism as the only difference; the TP8 arm collapses at 16 sessions, from 3,925 to 1,410 tok/s per GPU. The post names no precision and no speculation setting for the replay. The figure beside that text does not plot 541. Its axis counts prompt and generated tokens per GPU, and the DCP8 arm's 48-session point reads about 15,333 tok/s per GPU, about 122,660 across the 8 GPUs, at 8.9 tok/s per stream; 48 streams at 8.9 tok/s generate about 427 tok/s. 541 is neither that nor 122,660, and the post reconciles it with neither. The DCP8 curve peaks at 48 sessions and falls to about 12,750 at 56 and 8,520 at 64. Read from the SVG's data coordinates; the DCP8 panel has no axis labels of its own and is read on the TP8 panel's scale. [CORRECTED 2026-09-13: this row said 541 was roughly 2.4% of the same hardware's short-context batched figure. That set a generated-token rate for 8 GPUs against 22,464, a prompt-inclusive per-GPU figure multiplied by 8, and the row linked no hardware although it ran on GB300.]
785 tok/s per accelerator, counting prompt and generated tokens · SGLang v0.5.18 @ 71de97b2, TP8 · cookbook Low-Latency cell, random 8,192 in / 1,024 out, concurrency 16
vendor reported medium confidenceMarked Verified in the cookbook: P50 TTFT 3,539 ms, P50 TPOT 19.47 ms and 785 tok/s per GPU at concurrency 16, with the native MXFP4 checkpoint, no --kv-cache-dtype flag, so the cache stays at the model's bf16, and no speculation. The per-GPU figure counts prompt and generated tokens: 16 streams at those medians generate 16 × 1,024 / (3.539 + 1,024 × 0.01947) = 697.9 tok/s, and 785 × 8 × 1,024 / 9,216 = 697.8. Measured with --random-range-ratio 1.0, --warmup-requests 64 and --flush-cache, so no request reuses another's prefix.
1,395 tok/s per accelerator, counting prompt and generated tokens · SGLang v0.5.18 @ 71de97b2, TP8 + DCP8 · cookbook Balanced cell, random 8,192 in / 1,024 out, concurrency 64
vendor reported medium confidenceMarked Verified in the cookbook: P50 TTFT 11,635 ms, P50 TPOT 40.19 ms and 1,395 tok/s per GPU at concurrency 64, with the native MXFP4 checkpoint, decode context parallelism across the eight GPUs, a bf16 cache and no speculation. The per-GPU figure counts prompt and generated tokens: 64 × 1,024 / (11.635 + 1,024 × 0.04019) = 1,241.5 generated tok/s, against 1,395 × 8 × 1,024 / 9,216 = 1,240.0. The KDA state pool caps this recipe at 101 running requests, which is why the cookbook publishes no Balanced point past concurrency 64. Measured with --random-range-ratio 1.0, --warmup-requests 64 and --flush-cache.
1,987 tok/s per accelerator, counting prompt and generated tokens · SGLang v0.5.18 @ 71de97b2, TP8 + DCP8, DSPARK at a simulated acceptance length of 4.5 · cookbook Balanced DSPARK cell, random 8,192 in / 1,024 out, concurrency 64
vendor reported low confidenceMarked Verified in the cookbook, with a simulated acceptance: P50 TTFT 12,038 ms, P50 TPOT 24.47 ms and 1,987 tok/s per GPU at concurrency 64, the Balanced recipe with DSPARK speculative decoding and --max-running-requests 256. The cookbook pins the draft acceptance length with SGLANG_SIMULATE_ACC_LEN=4.5, so the cell reports what DSPARK's block of 7 delivers at that acceptance, not an acceptance rate measured on this workload, and it asks readers to measure against the same recipe without speculation before adopting it. SGLang pull request #39029, which added the MI350X cells, calls this cell's acceptance real in passing; the page's note, which names the setting and was on the page when that pull request merged, is followed here. Had the acceptance been real, it would have been measured on the benchmark's random prompts, on which the same pull request's real drafter accepted 7.2 to 7.4 of 8 draft tokens on MI350X, an acceptance it says real text will not reach. With DSPARK the KDA state pool caps admission at 68 running requests. The per-GPU figure counts prompt and generated tokens: 64 × 1,024 / (12.038 + 1,024 × 0.02447) = 1,766.7 generated tok/s, against 1,987 × 8 × 1,024 / 9,216 = 1,766.2.
462 tok/s per accelerator, counting prompt and generated tokens · SGLang v0.5.19 @ 12771786 (ROCm), TP8, fp8 KV · cookbook Balanced cell, random 8,192 in / 1,024 out, concurrency 16
vendor reported low confidenceMarked Final Verification In Progress in the cookbook: P50 TTFT 6,331 ms, P50 TPOT 32.8 ms and 462 tok/s per GPU at concurrency 16. The MI350X and MI355X share one recipe: TP8 on ROCm with AITER's A8W4 MoE kernels, Triton attention, an fp8_e4m3 KV cache and, in this cell, no speculation. The per-GPU figure counts prompt and generated tokens: 16 × 1,024 / (6.331 + 1,024 × 0.0328) = 410.4 generated tok/s, against 462 × 8 × 1,024 / 9,216 = 410.7. Not linked to a catalog system, which carries the MI355X and not the MI350X.
813 tok/s per accelerator, counting prompt and generated tokens · SGLang v0.5.19 @ 12771786 (ROCm), TP8, fp8 KV · cookbook Balanced cell, random 8,192 in / 1,024 out, concurrency 64
vendor reported low confidenceMarked Final Verification In Progress in the cookbook: P50 TTFT 18,665 ms, P50 TPOT 70.39 ms and 813 tok/s per GPU at concurrency 64, on the same recipe as the concurrency-16 cell. The per-GPU figure counts prompt and generated tokens: 64 × 1,024 / (18.665 + 1,024 × 0.07039) = 722.2 generated tok/s, against 813 × 8 × 1,024 / 9,216 = 722.7. SGLang pull request #39029, which added the MI350X cells, gives the running-request cap at the run's memory budget, --mem-fraction-static 0.85 with --context-length 16384, as 74 without speculation, above this cell's 64, and calls the concurrency-64 cells partly queue-bound without saying how. Not linked to a catalog system, which carries the MI355X and not the MI350X.
913 tok/s per accelerator, counting prompt and generated tokens · SGLang v0.5.19 @ 12771786 (ROCm), TP8, fp8 KV, DSPARK at a simulated acceptance length of 5 · cookbook Balanced DSPARK cell, random 8,192 in / 1,024 out, concurrency 64
vendor reported low confidenceMarked Final Verification In Progress in the cookbook: P50 TTFT 23,720 ms, P50 TPOT 31.45 ms and 913 tok/s per GPU at concurrency 64, with DSPARK speculative decoding on the shared MI350X/MI355X recipe. The acceptance was pinned, not measured. The page does not say so; SGLang pull request #39029, which added the MI350X cells, does: its DSPARK runs set SGLANG_SIMULATE_ACC_LEN=5 with SGLANG_SIMULATE_ACC_METHOD=match-expected, which accepts exactly 5 of the 8 draft tokens every round, so the figure is what the system delivers at that acceptance and not an acceptance any text reached, and the run's generated text was not used to judge quality. A run with the real drafter accepted 7.2 to 7.4 of 8 on the benchmark's random prompts and reached 1,261 tok/s per GPU at this concurrency; the pull request left it out as an acceptance real text will not reach. It gives the running-request cap at the run's memory budget, --mem-fraction-static 0.85 with --context-length 16384, as 48 with DSPARK and 74 without, so at most 48 of the 64 requests the client kept open ran at once, and it calls the concurrency-64 cells partly queue-bound. Here the medians do not reproduce the per-GPU figure: 64 × 1,024 / (23.720 + 1,024 × 0.03145) = 1,171.9 generated tok/s, against 913 × 8 × 1,024 / 9,216 = 811.6, a ratio of 0.69. The cells that do close run below their caps: the same recipe's concurrency-16 DSPARK cell, 864 tok/s per GPU, agrees within 2% at 16 requests against 48, and the concurrency-64 cell without speculation within 0.1% at 64 against 74. The per-GPU figure is a whole-run total and the medians describe a typical request; the cookbook does not say which to trust. Not linked to a catalog system, which carries the MI355X and not the MI350X.
NVIDIA · accessed 2026-07-19 primary
Open source ↗NVIDIA · accessed 2026-09-09 primary
Open source ↗NVIDIA Newsroom · accessed 2026-07-19 primary
Open source ↗NVIDIA · accessed 2026-09-25 primary
Open source ↗NVIDIA · accessed 2026-09-25 primary
Open source ↗Mercatus AI · accessed 2026-07-19 independent research
Open source ↗NVIDIA · accessed 2026-07-19 primary
Open source ↗NVIDIA · accessed 2026-07-19 primary
Open source ↗Reuters (Investing.com reprint) · accessed 2026-07-19 independent research
Open source ↗NVIDIA · accessed 2026-07-19 primary
Open source ↗NVIDIA · accessed 2026-07-19 primary
Open source ↗AMD · accessed 2026-07-19 primary
Open source ↗AMD · accessed 2026-07-19 primary
Open source ↗AMD · accessed 2026-09-25 primary
Open source ↗AMD · accessed 2026-09-13 primary
Open source ↗Moonshot AI · accessed 2026-07-24 primary
Open source ↗Meta AI · accessed 2026-08-21· published 2024-07-31 primary
Open source ↗Meta · accessed 2026-07-19 primary
Open source ↗Mistral AI · accessed 2026-10-07· published 2025-12-02 primary
Open source ↗Mistral AI (Hugging Face) · accessed 2026-07-19 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗Google · accessed 2026-09-26 primary
Open source ↗Google · accessed 2026-09-26 primary
Open source ↗xAI · accessed 2026-09-25 primary
Open source ↗xAI · accessed 2026-09-25 primary
Open source ↗Mistral AI · accessed 2026-09-25 primary
Open source ↗Mistral AI · accessed 2026-09-25 primary
Open source ↗MLCommons · accessed 2026-07-19 primary
Open source ↗MLCommons · accessed 2026-09-13 primary
Open source ↗AMD ROCm · accessed 2026-09-13 primary
Open source ↗MLCommons · accessed 2026-07-24 primary
Open source ↗MLCommons · accessed 2026-10-07 primary
Open source ↗MLCommons · accessed 2026-07-24 primary
Open source ↗MLCommons (LoadGen) · accessed 2026-09-13 primary
Open source ↗MLCommons (endpoints harness) · accessed 2026-10-07 primary
Open source ↗AMD ROCm · accessed 2026-07-24 primary
Open source ↗Nebius · accessed 2026-07-24 primary
Open source ↗NVIDIA · accessed 2026-07-24 primary
Open source ↗SemiAnalysis · accessed 2026-07-24 independent research
Open source ↗vLLM Project · accessed 2026-07-24 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Qwen (Hugging Face) · accessed 2026-08-29 primary
Open source ↗Google · accessed 2026-07-24 primary
Open source ↗DeepSeek (Hugging Face) · accessed 2026-07-24 primary
Open source ↗DeepSeek (Hugging Face) · accessed 2026-07-24 primary
Open source ↗Xiaomi · accessed 2026-10-01· published 2026-04-28 primary
Open source ↗Xiaomi (Hugging Face) · accessed 2026-07-24 primary
Open source ↗Z.ai (Hugging Face) · accessed 2026-10-07· published 2026-06-16 primary
Open source ↗Moonshot AI (Hugging Face) · accessed 2026-10-07· published 2026-04-20 primary
Open source ↗DeepSeek · accessed 2026-09-25 primary
Open source ↗Xiaomi · accessed 2026-09-25 primary
Open source ↗Alibaba Cloud · accessed 2026-10-07 primary
Open source ↗Alibaba Cloud · accessed 2026-10-07 primary
Open source ↗Z.ai · accessed 2026-09-25 primary
Open source ↗Intel · accessed 2026-07-24 primary
Open source ↗ServeTheHome · accessed 2026-07-24 independent research
Open source ↗Anthropic · accessed 2026-08-01· published 2026-07-24 primary
Open source ↗Anthropic · accessed 2026-08-01· published 2026-06-30 primary
Open source ↗OpenAI · accessed 2026-08-01· published 2026-07-30 primary
Open source ↗OpenAI · accessed 2026-09-25· published 2026-08-21 primary
Open source ↗Moonshot AI (Hugging Face) · accessed 2026-09-13· published 2026-07-27 primary
Open source ↗DeepSeek (Hugging Face) · accessed 2026-08-01· published 2026-07-31 primary
Open source ↗DeepSeek · accessed 2026-09-25 primary
Open source ↗Meta (Hugging Face) · accessed 2026-08-01· published 2025-04-05 primary
Open source ↗Meta · accessed 2026-08-01 primary
Open source ↗vLLM · accessed 2026-08-01· published 2026-07-27 primary
Open source ↗LMSYS Org · accessed 2026-09-25· published 2026-07-27 primary
Open source ↗SGLang · accessed 2026-09-25 primary
Open source ↗SGLang (pull request #39029) · accessed 2026-09-25· published 2026-09-11 primary
Open source ↗LG AI Research (Hugging Face) · accessed 2026-08-01· published 2026-07-31 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-08-01· published 2026-06-04 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-08-01· published 2026-03-11 primary
Open source ↗MiniMax (Hugging Face) · accessed 2026-10-01· published 2026-06-12 primary
Open source ↗Tencent (Hugging Face) · accessed 2026-08-01· published 2026-07-06 primary
Open source ↗Cohere (Hugging Face) · accessed 2026-10-01 primary
Open source ↗DeepSeek · accessed 2026-08-14· published 2026-08-13 primary
Open source ↗DeepSeek (Hugging Face) · accessed 2026-08-14· published 2026-08-13 primary
Open source ↗xAI · accessed 2026-08-14· published 2026-08-12 primary
Open source ↗Meta (Hugging Face) · accessed 2026-08-14· published 2026-08-10 primary
Open source ↗Meta · accessed 2026-09-25 primary
Open source ↗Meta · accessed 2026-09-25 primary
Open source ↗Alibaba Cloud Model Studio · accessed 2026-10-07 primary
Open source ↗Qwen (Hugging Face) · accessed 2026-10-01 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-10-07· published 2026-08-11 primary
Open source ↗Qwen (Hugging Face) · accessed 2026-10-01· published 2026-08-14 primary
Open source ↗Google DeepMind · accessed 2026-08-29· published 2026-08-13 primary
Open source ↗Artificial Analysis · accessed 2026-10-07 independent research
Open source ↗Artificial Analysis · accessed 2026-09-25 independent research
Open source ↗Google · accessed 2026-08-01 primary
Open source ↗IntuitionLabs · accessed 2026-08-01 independent research
Open source ↗NVIDIA · accessed 2026-08-01 primary
Open source ↗HPE · accessed 2026-08-01 independent research
Open source ↗ServeTheHome · accessed 2026-08-01· published 2026-01-05 independent research
Open source ↗SemiAnalysis · accessed 2026-08-01· published 2026-02-25 independent research
Open source ↗NVIDIA · accessed 2026-09-26· published 2026-01-05 primary
Open source ↗NVIDIA · accessed 2026-09-26 primary
Open source ↗NVIDIA · accessed 2026-09-26 primary
Open source ↗NVIDIA · accessed 2026-09-26· published 2026-07-21 primary
Open source ↗NVIDIA · accessed 2026-09-26· published 2026-08-26 primary
Open source ↗NVIDIA · accessed 2026-09-26· published 2026-09-15 primary
Open source ↗NVIDIA · accessed 2026-09-26· published 2026-09-16 primary
Open source ↗MLCommons · accessed 2026-09-26 primary
Open source ↗AMD Newsroom · accessed 2026-08-01· published 2026-07-23 primary
Open source ↗The Register · accessed 2026-08-01· published 2026-07-23 independent research
Open source ↗AMD · accessed 2026-09-26 primary
Open source ↗AMD · accessed 2026-09-26· published 2026-08-05 primary
Open source ↗AMD · accessed 2026-09-26· published 2026-07-20 primary
Open source ↗AMD · accessed 2026-09-26 primary
Open source ↗AMD · accessed 2026-09-26 primary
Open source ↗AMD · accessed 2026-09-26 primary
Open source ↗AMD Newsroom · accessed 2026-09-26· published 2026-07-23 primary
Open source ↗AMD · accessed 2026-09-26· published 2025-06-12 primary
Open source ↗Red Hat · accessed 2026-08-01 primary
Open source ↗Z.ai · accessed 2026-08-15· published 2026-08-14 primary
Open source ↗Z.ai · accessed 2026-09-25 primary
Open source ↗Z.ai (Hugging Face) · accessed 2026-08-15 primary
Open source ↗Qwen (Hugging Face) · accessed 2026-08-15 primary
Open source ↗Qwen (Hugging Face) · accessed 2026-08-15 primary
Open source ↗MiniMax (Hugging Face) · accessed 2026-08-15 primary
Open source ↗Mistral AI (Hugging Face) · accessed 2026-08-15 primary
Open source ↗Mistral AI (Hugging Face) · accessed 2026-08-15 primary
Open source ↗Google (Hugging Face) · accessed 2026-08-15 primary
Open source ↗Meta (Hugging Face) · accessed 2026-08-15 primary
Open source ↗DeepSeek · accessed 2026-08-15· published 2026-04-26 primary
Open source ↗OpenAI · accessed 2026-08-27 primary
Open source ↗Anthropic · accessed 2026-08-15 primary
Open source ↗Thomas Farfeleder · accessed 2026-08-15 independent research
Open source ↗AtomicChat (Hugging Face) · accessed 2026-10-07 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-10-07 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-10-07 primary
Open source ↗AtomicChat (Hugging Face) · accessed 2026-10-07 primary
Open source ↗Google (Hugging Face) · accessed 2026-10-01 primary
Open source ↗Google (Hugging Face) · accessed 2026-10-01 primary
Open source ↗Z.ai (Hugging Face) · accessed 2026-10-07· published 2026-08-26 primary
Open source ↗Z.ai (Hugging Face) · accessed 2026-08-27 primary
Open source ↗Z.ai · accessed 2026-09-25 primary
Open source ↗Hugging Face · accessed 2026-08-27 primary
Open source ↗Qwen (Hugging Face) · accessed 2026-08-27 primary
Open source ↗Qwen (Hugging Face) · accessed 2026-08-27 primary
Open source ↗Alibaba · accessed 2026-10-07 primary
Open source ↗Hugging Face · accessed 2026-08-27 primary
Open source ↗vLLM · accessed 2026-10-07 primary
Open source ↗SGLang · accessed 2026-10-07 primary
Open source ↗vLLM · accessed 2026-10-07 primary
Open source ↗SGLang · accessed 2026-10-07 primary
Open source ↗SemiAnalysis · accessed 2026-10-07 independent research
Open source ↗vLLM · accessed 2026-08-27 primary
Open source ↗vLLM · accessed 2026-09-26 primary
Open source ↗vLLM (pull request #55557) · accessed 2026-09-26· published 2026-09-16 primary
Open source ↗vLLM · accessed 2026-09-26 primary
Open source ↗vLLM · accessed 2026-09-26· published 2026-09-22 primary
Open source ↗SGLang (pull request #38855) · accessed 2026-09-26· published 2026-09-10 primary
Open source ↗SGLang (pull request #38851) · accessed 2026-09-26· published 2026-09-11 primary
Open source ↗SGLang · accessed 2026-09-26· published 2026-09-18 primary
Open source ↗SGLang · accessed 2026-09-26 primary
Open source ↗Sebastian Raschka · accessed 2026-08-27· published 2026-08-26 independent research
Open source ↗Sebastian Raschka · accessed 2026-08-27· published 2026-08-11 independent research
Open source ↗Qwen (Hugging Face) · accessed 2026-08-27 primary
Open source ↗Qwen (Hugging Face) · accessed 2026-08-27 primary
Open source ↗Xiaomi (Hugging Face) · accessed 2026-08-27 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Unsloth AI (Hugging Face) · accessed 2026-08-29 secondary
Open source ↗Red Hat (Hugging Face) · accessed 2026-08-29 secondary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-10-07 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-10-07 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-10-07 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-10-07 primary
Open source ↗NVIDIA (Hugging Face) · accessed 2026-10-07 primary
Open source ↗Z.ai (Hugging Face) · accessed 2026-10-01 primary
Open source ↗Z.ai (Hugging Face) · accessed 2026-08-29 primary
Open source ↗Z.ai (Hugging Face) · accessed 2026-08-29 primary
Open source ↗Google · accessed 2026-09-25 primary
Open source ↗Google · accessed 2026-09-25 primary
Open source ↗Google · accessed 2026-09-25 primary
Open source ↗Tencent (Hugging Face) · accessed 2026-08-29· published 2026-08-28 primary
Open source ↗Tencent (Hugging Face) · accessed 2026-09-09· published 2026-08-27 primary
Open source ↗IBM (Hugging Face) · accessed 2026-10-07· published 2026-08-25 primary
Open source ↗InclusionAI (Hugging Face) · accessed 2026-10-07· published 2026-08-04 primary
Open source ↗InclusionAI (Hugging Face) · accessed 2026-09-26· published 2026-05-14 primary
Open source ↗InclusionAI (Hugging Face) · accessed 2026-09-26· published 2026-02-10 primary
Open source ↗InclusionAI (Hugging Face) · accessed 2026-09-26· published 2025-10-10 primary
Open source ↗Ling Team, Inclusion AI (arXiv:2606.15079) · accessed 2026-09-26· published 2026-06-13 primary
Open source ↗Ant Group · accessed 2026-09-26 primary
Open source ↗vLLM · accessed 2026-09-26· published 2026-05-14 primary
Open source ↗Ant Group · accessed 2026-09-26 primary
Open source ↗StepFun (Hugging Face) · accessed 2026-10-07· published 2026-05-29 primary
Open source ↗Mistral AI (Hugging Face) · accessed 2026-09-11· published 2026-03-16 primary
Open source ↗Google (Hugging Face) · accessed 2026-08-29· published 2026-06-04 primary
Open source ↗NVIDIA · accessed 2026-09-08 primary
Open source ↗NVIDIA · accessed 2026-09-08 primary
Open source ↗NVIDIA · accessed 2026-09-25 primary
Open source ↗NVIDIA · accessed 2026-09-25 primary
Open source ↗NVIDIA · accessed 2026-09-08 primary
Open source ↗NVIDIA · accessed 2026-09-09 primary
Open source ↗Lenovo Press · accessed 2026-09-08 secondary
Open source ↗Tom's Hardware · accessed 2026-09-08 secondary
Open source ↗NVIDIA · accessed 2026-09-08 primary
Open source ↗NVIDIA · accessed 2026-09-08 primary
Open source ↗NVIDIA · accessed 2026-09-08 primary
Open source ↗ServeTheHome · accessed 2026-09-25· published 2025-10-14 primary
Open source ↗AMD · accessed 2026-09-08 primary
Open source ↗TechPowerUp · accessed 2026-09-08 secondary
Open source ↗AMD · accessed 2026-09-26 primary
Open source ↗AMD · accessed 2026-09-26 primary
Open source ↗AMD · accessed 2026-09-26· published 2026-05-20 primary
Open source ↗AMD · accessed 2026-09-26· published 2026-07-06 primary
Open source ↗Apple · accessed 2026-09-25 primary
Open source ↗Apple · accessed 2026-09-25· published 2026-08-25 primary
Open source ↗Apple · accessed 2026-09-25· published 2026-09-21 primary
Open source ↗Apple · accessed 2026-09-25 primary
Open source ↗Apple · accessed 2026-09-25 primary
Open source ↗Apple · accessed 2026-09-25 primary
Open source ↗Apple · accessed 2026-09-25· published 2026-08-25 primary
Open source ↗Apple · accessed 2026-09-25· published 2026-09-22 primary
Open source ↗Apple · accessed 2026-09-25· published 2026-09-21 primary
Open source ↗Apple · accessed 2026-09-25 primary
Open source ↗Apple · accessed 2026-09-25 primary
Open source ↗arXiv · accessed 2026-09-25 independent research
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗Google · accessed 2026-09-08 primary
Open source ↗Google DeepMind · accessed 2026-09-13 primary
Open source ↗OpenRouter · accessed 2026-09-08 secondary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗DeepSeek (Hugging Face) · accessed 2026-09-11· published 2026-09-10 primary
Open source ↗DeepSeek (Hugging Face) · accessed 2026-09-11· published 2026-09-10 primary
Open source ↗DeepSeek AI (ISCA '25) · accessed 2026-09-11· published 2025-05-14 independent research
Open source ↗Hugging Face · accessed 2026-09-11 primary
Open source ↗DeepSeek · accessed 2026-09-25· published 2026-09-10 primary
Open source ↗Google · accessed 2026-09-25 primary
Open source ↗Google · accessed 2026-09-13 primary
Open source ↗Z.ai · accessed 2026-09-13 primary
Open source ↗Z.ai · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25· published 2026-09-22 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗Anthropic · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-13 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25· published 2026-09-22 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗OpenAI · accessed 2026-09-25 primary
Open source ↗Google · accessed 2026-09-13 primary
Open source ↗Google · accessed 2026-09-26 primary
Open source ↗Google · accessed 2026-09-13 primary
Open source ↗Google · accessed 2026-09-13 primary
Open source ↗Moonshot AI · accessed 2026-09-25 primary
Open source ↗Moonshot AI · accessed 2026-09-25 primary
Open source ↗xAI · accessed 2026-09-13 primary
Open source ↗xAI · accessed 2026-09-25 primary
Open source ↗xAI · accessed 2026-09-25 primary
Open source ↗xAI · accessed 2026-09-25· published 2026-09-21 primary
Open source ↗xAI · accessed 2026-09-25 primary
Open source ↗xAI · accessed 2026-09-25 primary
Open source ↗Alibaba Cloud Model Studio · accessed 2026-10-07 primary
Open source ↗Alibaba Cloud Model Studio · accessed 2026-10-07 primary
Open source ↗Alibaba Cloud Model Studio · accessed 2026-10-07 primary
Open source ↗DeepSeek · accessed 2026-09-13 primary
Open source ↗Xiaomi · accessed 2026-09-25 primary
Open source ↗Xiaomi · accessed 2026-09-25 primary
Open source ↗Xiaomi · accessed 2026-09-25 primary
Open source ↗Xiaomi · accessed 2026-09-25 primary
Open source ↗Xiaomi · accessed 2026-09-25 primary
Open source ↗Xiaomi · accessed 2026-09-25 primary
Open source ↗Xiaomi (Hugging Face) · accessed 2026-09-26 primary
Open source ↗Xiaomi (Hugging Face) · accessed 2026-09-26 primary
Open source ↗Xiaomi (Hugging Face) · accessed 2026-09-26 primary
Open source ↗Xiaomi · accessed 2026-09-25 primary
Open source ↗Mistral AI · accessed 2026-09-25 primary
Open source ↗Mistral AI · accessed 2026-09-13 primary
Open source ↗Qwen (GitHub) · accessed 2026-10-07 primary
Open source ↗DeepSeek · accessed 2026-10-01· published 2026-04-24 primary
Open source ↗Cohere · accessed 2026-10-01· published 2026-05-20 primary
Open source ↗MiniMax · accessed 2026-10-01· published 2026-06-01 primary
Open source ↗Google · accessed 2026-10-01· published 2026-04-02 primary
Open source ↗Google · accessed 2026-10-01 primary
Open source ↗StepFun · accessed 2026-10-07· published 2026-05-29 primary
Open source ↗Ant Group (Business Wire, read as Interfax Newswire’s copy) · accessed 2026-10-07· published 2026-07-27 primary
Open source ↗IT之家 · accessed 2026-10-07· published 2026-08-08 secondary
Open source ↗Mistral AI · accessed 2026-10-07· published 2026-03-16 primary
Open source ↗Mistral AI · accessed 2026-10-07· published 2025-12-02 primary
Open source ↗Xiaomi · accessed 2026-10-07· published 2026-09-22 primary
Open source ↗Meta · accessed 2026-10-07· published 2024-07-23 primary
Open source ↗Meta (GitHub) · accessed 2026-10-07· published 2024-07-23 primary
Open source ↗Ant Ling (X) · accessed 2026-10-07· published 2026-08-04 primary
Open source ↗Xiaomi MiMo (X) · accessed 2026-10-07· published 2026-09-22 primary
Open source ↗Kimi.ai (X) · accessed 2026-10-07· published 2026-04-20 primary
Open source ↗Z.ai · accessed 2026-10-07 primary
Open source ↗IT之家 · accessed 2026-10-07· published 2026-06-17 secondary
Open source ↗One row per claim in the catalog: what it is about, how strong the evidence is, which sources back it, and — where the figure was derived rather than read off a page — the derivation itself, so you can redo it and disagree. The five grades are defined under Methodology.
563 of 563 claims · 109 are derived or assumed
Official specifications.
0.7 kW is the accelerator alone: NVIDIA rates the H100 SXM at "Up to 700W (configurable)". An SXM module runs inside an eight-GPU server, so the 0.575 kW host allowance is one eighth of the host share of NVIDIA's own eight-GPU Hopper system. NVIDIA's DGX H200 datasheet gives 10.2 kW maximum system power for its standard configuration, and its eight H200s are rated "Up to 700W" each on NVIDIA's H200 page: 5.6 kW of accelerators, which leaves 4.6 kW for the CPUs, memory, NVSwitch chips, networking, fans and power-supply losses.
Memory/bandwidth use official H200 values and the 8-GPU price range is independent research. REGRADED 2026-09-25 from assumption: the assumption was the power figure, which now has its own record citing NVIDIA's DGX H200 datasheet, so what this record still rests on is a price estimate.
NVIDIA's DGX H200 datasheet: "System power usage 10.2kW max" for the standard configuration, host included; its Custom Thermal Solution variant is specified up to 14.3 kW. This entry prices a generic eight-GPU H200 server, so the figure is NVIDIA's reference system rather than any one OEM build. It carried the same 10.2 kW before 2026-09-25, recorded as a planning assumption with no source.
Official node specifications; price intentionally missing.
The DGX B200 User Guide's power specification gives "14.3 kW max." for the system, drawn through six 3.3 kW supplies in 5+1 redundancy, with each GPU capped at 1,000 W. The supplies' combined rating is capacity, not draw.
[CORRECTED at the 2026-08-01 refresh. This entry previously carried $550,000, a figure that re-verification could not trace to any source; current independent reporting puts an 8-GPU B300 node at $300k-350k baseline and up to $400-500k from resellers, so the point value is now $350k with the spread recorded.] Memory (2,304 GB), 64 TB/s aggregate HBM bandwidth and 14.5 kW are all exact matches to NVIDIA's own DGX B300 User Guide. Note that the 14.4 TB/s figure circulating in secondary write-ups is the NVLink switch fabric, not memory bandwidth. Deployment multiplier is an editable planning assumption.
The DGX B300 User Guide lists "Power consumption: 14.5 kW" for the system and specifies its AC and DC inputs for "15 kW system max."; the twelve 3.2 kW supplies are capacity, not draw. Read in the guide's HTML edition on 2026-09-25.
[CORRECTED at the 2026-08-01 refresh: power is now NVIDIA's own approximately 120 kW from the DGX GB200 NVL72 User Guide, replacing the 132 kW previously recorded.] CONFLICT preserved: infrastructure partners co-designing reference racks with NVIDIA (Vertiv, Schneider Electric, HPE) provision for up to 132 kW nominal with far higher transient peaks, so 132 kW is the right number for facility sizing and 120 kW for the rack itself. Memory and fabric are official; the 13.4 TB rack aggregate is NVIDIA's own published figure and deliberately is not 72 x a per-GPU value.
The DGX GB200 User Guide: "The rack power consumption is approximately 120kW." That is the whole rack: compute trays, NVLink switch trays and the eight power shelves feeding them. Partners provision up to 132 kW, which is facility capacity rather than draw.
[CORRECTED at the 2026-08-01 refresh. The previous $3.0M simply reused the GB200 estimate, while reporting consistently puts GB300 at a premium over its predecessor ($3.0-4.0M, with one analyst note derived from a large customer order suggesting $3.7-4.0M); the midpoint is now recorded with the spread. Power moved from 132 kW to 135 kW: NVIDIA publishes no rack figure for GB300 at all, OEM reference documentation from HPE and Lenovo converges on 132-135 kW nominal, and other trackers report 140-142 kW — treat 132-142 kW as the honest range.] Memory is derived as 72 x 288 GB; NVIDIA's own page rounds it to 20 TB.
NVIDIA publishes no rack power for GB300 NVL72. OEM reference documentation from HPE and Lenovo converges on 132-135 kW nominal for the whole rack and other trackers report 140-142 kW; 135 kW is a point in that range, not a vendor figure.
NAMING: NVIDIA announced this rack as "NVL144" at GTC 2025 counting compute dies, then as "NVL72" at CES 2026 counting GPU packages. They are the same rack, not two products. Memory is 72 × 288 GB of HBM4, which NVIDIA's datasheet and product page both give as 20.7 TB. Bandwidth is the 1,580 TB/s the datasheet prints for the rack; 72 × its 22 TB/s per GPU would be 1,584 TB/s, and the printed rack figure is the one stored. The 260 TB/s NVLink 6 fabric is the datasheet's and NVIDIA's CES release's. CONFLICT preserved: NVIDIA's product page, read 2026-09-26, gives 19.2 TB/s per GPU and 1,400 TB/s for the rack, and 3 TB/s of NVLink per GPU and 216 TB/s for the rack. The same page's "100 MW AI Factory Specifications" give "12 PB HBM4 | 800 PB/s" for 40K GPUs, 20 TB/s per GPU, which fits the page's own 19.2 TB/s, 768 PB/s across 40,000 GPUs, and not the datasheet's 22 TB/s, 880 PB/s; it is the same page as the 19.2, so it adds no independent document to that side. The datasheet that page links calls its specifications "Preliminary information" whose values are "up to and subject to change", and neither document says which describes the racks now shipping. The stored figures are the datasheet's, which every earlier report also gave, and confidence is low because NVIDIA's own two documents disagree. The 190 kW power draw is NOT vendor-published — it is the low end of a supply-chain analyst range of roughly 190 kW (Max-Q) to 230 kW (Max-P); the Power draw field under Operations replaces it with a quote. SHIPPING: at CES on 2026-01-05 NVIDIA said "Rubin-based products will be available from partners the second half of 2026". On 2026-07-21 it said racks were running at CoreWeave, Google Cloud, Microsoft Azure, Oracle Cloud Infrastructure and Nebius, and an update on the same post, headed "Tuesday, July 24, 1:30 p.m. PT", says Nebius "got early access". That header's weekday and date disagree: 2026-07-24 is a Friday, and the Tuesdays either side are 2026-07-21, the post's own date, and 2026-07-28, the last change its metadata records. Its investor presentation of 2026-08-26 says NVIDIA "Commenced production shipments of Vera Rubin in early August", which is 2026-08-01 to 2026-08-10 read as the first third of the month, and on 2026-09-15 it said "Vera Rubin is in full production and scaling across the ecosystem". It entered the rack in MLPerf Inference v6.1, published 2026-09-16, as a preview system rather than an available one; MLCommons calls a system available only once it is priced, has shipped to a third party and can be bought by further third parties without early-access approval. So the rack ships, NVIDIA has not said it is generally available, and it has published no price: this catalog's preview status, released but not generally available. [CORRECTED 2026-09-26: memory bandwidth was stored as 1,584 TB/s, 72 × the datasheet's 22 TB/s per GPU, a rack figure NVIDIA does not print. The datasheet prints 1,580 TB/s for the rack, and where a vendor prints a rack figure this catalog stores it, as it does AMD's 1,670 TB/s for Helios.] [CORRECTED 2026-09-26: the entry was announced, which this catalog defines as not yet shipping, and stored 2026-07-01 as the day it became available. That is the first day of the half NVIDIA named at CES, not a day NVIDIA gave. The note said "Declared in full production at CES 2026 with volume availability in the second half of 2026", citing only ServeTheHome and SemiAnalysis.] [UPGRADED 2026-09-26: estimated to official-claim, at low confidence still. Memory, bandwidth and fabric are now read from NVIDIA's datasheet, product page and CES release rather than from two outlets' reports of them, so the reason for the downgrade below no longer holds.] [DOWNGRADED 2026-08-29: this record was graded official-claim at medium confidence while citing no primary source at all — both sourceIds are independent-research, ServeTheHome and SemiAnalysis, so nothing here was read from NVIDIA. The gb300-nvl72 record carries the identical caveat about an unpublished rack power figure, cites one primary source more than this one, and is graded estimated at low confidence; the entry with the weaker evidence held the stronger grade. Nothing about the figures changed — 288 GB HBM4 x 72 and the NVLink 6 fabric are still what NVIDIA announced at CES 2026 and what those two outlets reported — only the claim this catalog makes about how it knows them.]
NVIDIA has published no rack power for Vera Rubin NVL72: its datasheet, its product page and its MLPerf Inference v6.1 system description, read on 2026-09-26, state none. 190 kW is the low end of a supply-chain analyst range of roughly 190 kW (Max-Q) to 230 kW (Max-P) for the whole rack. The system description does rate this rack's GPUs, "2300W TGP Per GPU": 165.6 kW for the 72 GPUs alone, before the 36 Vera CPUs, the NVLink switch trays and the networking, so 190 kW is not below the GPUs' own rating. The product page gives a cross-check: its "100 MW AI Factory Specifications" put "40K GPUs" in 100 MW, footnoted "Specifications based on an at-scale AI factory using NVIDIA DSX with MaxLPS", which is 2.5 kW per GPU for everything those 100 MW cover, about 180 kW per 72 GPUs and 5% under the stored figure. The page does not say whether the 100 MW is the IT load or the whole facility's power, cooling included, and does not define MaxLPS, so the figure cannot replace the stored one: the calculator applies PUE separately, and a figure that may already carry a facility's overhead is not a rack's own draw. The Power draw field under Operations replaces it with a quote.
NAMING: AMD's own press releases call this generation both "MI450 Series" (the OpenAI and Anthropic partnership announcements) and "MI400 Series" with the MI455X as flagship (the July 2026 launch). Same hardware. 432 GB HBM4 per accelerator across 72 accelerators is a vendor figure, as is the 260 TB/s UALink-over-Ethernet fabric; AMD's MI455X and Helios pages state them. BANDWIDTH: the stored 1,670 TB/s is AMD's own rack figure. Its MI400 Series page says Helios offers "1.67 PB/s of memory bandwidth", and a tile on that page and AMD's Helios launch release of 2026-07-23 round it to 1.7 PB/s. The 23.3 TB/s per GPU that the MI455X page, the MI400 Series page and the Helios page's specification table give agrees with it: 72 × 23.3 TB/s is 1,677.6 TB/s, within 0.5%. CONFLICT preserved on bandwidth: the compute-tray section of the same Helios page gives the MI455X "19.6 TB/s memory bandwidth per GPU", which would make the rack 1,411.2 TB/s. That is the figure AMD gave when it previewed the MI400 Series on 2025-06-12, and it cannot produce the launch release's claim of "6% more HBM bandwidth" than the leading competitive solution, which its footnote names as NVIDIA's Vera Rubin NVL72: AMD's pages set 23.3 TB/s against 22.0 TB/s for Vera Rubin, 5.9% more, where 19.6 TB/s would be 11% less. So the entry carries 1.67 PB/s, graded as AMD's claim at medium confidence: apart from that one section and the 2025 preview, every AMD statement read here agrees with it. AMD's product pages show no date; their metadata gives 2026-09-01 as their last update, and they were read on 2026-09-26. CONFLICT preserved on power: coverage of the same launch event cites both roughly 140 kW and 225-245 kW of bus-bar capacity under load; AMD has published no single authoritative rack figure, so 190 kW is recorded as a midpoint planning assumption, and the Power draw field under Operations replaces it with whatever a vendor quote states. AMD does not sell the rack itself: its Form 10-Q for the quarter to 2026-06-27 says it supplies products used in a Helios system and does not manufacture or sell the completed rack, which customers' manufacturing partners build. When AMD said it would ship is its release window. [CORRECTED 2026-09-26: this said the rack was "declared in full production 2026-07-23 with shipments from Q3 2026", a paraphrase that cited no AMD source, and the entry stored 2026-09-01 as the day it became available, a day no AMD page names.]
AMD has published no rack figure. Coverage of the same launch cites roughly 140 kW and 225-245 kW of bus-bar capacity under load; 190 kW is a midpoint this repository chose between them, for the whole rack. The Power draw field under Operations replaces it with whatever a vendor quote states.
Lisa Su said it on AMD's second-quarter earnings call and closed the same remarks with “initial Helios shipments set to begin this quarter”. That quarter is AMD's fiscal third. AMD's fiscal year ends on the last Saturday in December, 2026-12-26 this year, 52 weeks, and its Form 10-Q gives the second quarter as the 13 weeks to 2026-06-27, so the third quarter's 13 weeks end on 2026-09-26, four days before the calendar quarter. The window runs from the day she said it to that date. Two wider statements agree with it: the 10-Q expects Helios “to start shipping in the second half of fiscal year 2026”, and AMD's release with Microsoft of 2026-07-20 says “AMD will begin shipping Helios to customers, including Microsoft, in the second half of 2026.” AMD has announced no Helios shipment since: neither its newsroom, read to its item of 2026-09-24, nor its investor-relations releases, the latest of 2026-08-31, reports one.
Memory and bandwidth are official. REGRADED 2026-09-25 from assumption: that grade was the 8.5 kW full-node planning value, and power now carries its own record, the data sheet's 6 kW for eight GPUs plus a cited host allowance.
6 kW is the accelerators alone: eight MI300X at the 750 W maximum board power of AMD's platform data sheet. The 4.6 kW host allowance comes from the eight-GPU system whose accelerators are closest to 750 W. NVIDIA's DGX H200 datasheet gives 10.2 kW maximum system power for its standard configuration, and its eight H200s are rated "Up to 700W" each on NVIDIA's H200 page: 5.6 kW of accelerators, which leaves 4.6 kW for the CPUs, memory, NVSwitch chips, networking, fans and power-supply losses. Four NVSwitch chips are part of that share and this platform has none, so it errs slightly high here. It replaces an 8.5 kW full-node planning value whose 2.5 kW for the host had no source.
Memory and bandwidth are official. REGRADED 2026-09-25 from assumption: that grade was the 10.5 kW full-node planning value, and power now carries its own record, the data sheet's 8 kW for eight GPUs plus a cited host allowance.
8 kW is the accelerators alone: eight MI325X at the 1,000 W maximum of AMD's platform data sheet. The 6.3 kW host allowance comes from the eight-GPU system whose accelerators draw the same 1,000 W. NVIDIA's DGX B200 User Guide gives 14.3 kW maximum system power and caps each of its eight GPUs at 1,000 W: 8.0 kW of accelerators, which leaves 6.3 kW for everything else in the node. It replaces a 10.5 kW full-node planning value whose 2.5 kW for the host had no source.
Official GPU specifications; the node price is a planning assumption. Power carries its own record since 2026-09-25, AMD's 1,400 W per GPU plus a cited host allowance, in place of a 14 kW planning value.
11.2 kW is the accelerators alone: eight MI355X at the 1,400 W typical board power on AMD's product page. The 6.3 kW host allowance comes from the highest-power eight-GPU system for which a vendor publishes both figures. NVIDIA's DGX B200 User Guide gives 14.3 kW maximum system power and caps each of its eight GPUs at 1,000 W: 8.0 kW of accelerators, which leaves 6.3 kW for everything else in the node. The MI355X is the liquid-cooled part, and an air-cooled host spends part of that share on fans, so the allowance likely errs high here. It replaces a 14 kW planning value whose 2.8 kW for the host had no source.
128 GB HBM2e and 900 W per accelerator are official. REGRADED 2026-09-25 from assumption: that grade was a 9.5 kW full-node planning value, and power now carries its own record, the 7.2 kW the eight accelerators draw plus a cited host allowance.
7.2 kW is the accelerators alone: eight Gaudi 3 at 900 W each. The 6.3 kW host allowance comes from the eight-GPU system whose accelerators are closest to 900 W. NVIDIA's DGX B200 User Guide gives 14.3 kW maximum system power and caps each of its eight GPUs at 1,000 W: 8.0 kW of accelerators, which leaves 6.3 kW for everything else in the node. It replaces a 9.5 kW full-node planning value whose 2.3 kW for the host had no source.
96 GB GDDR7 ECC, 1,792 GB/s and 600 W are NVIDIA's own specifications, and so is the price, $16,000.00 on NVIDIA's own store. Until 2026-09-25 this entry said NVIDIA published no price and carried a range, $8,000 to $16,000, from the March 2025 pre-orders to press reporting of the September 2026 street price. Beside the vendor's one price, the low end describes what the card cost eighteen months earlier rather than a price on offer, so no range is carried. Carries no NVLink: the datasheet lists PCIe 5.0 x16 as the whole of its system interface, and the Server Edition's product brief states 'NVIDIA NVLink: Not supported' outright for the same silicon. NVLink left this product line two generations ago — the RTX A6000 had a bridge for two cards at 112 GB/s, the RTX 6000 Ada that replaced it did not — so a second card in the same box talks over the host bus at 128 GB/s, not over a fabric.
0.6 kW is the card alone, its 600 W maximum on NVIDIA's datasheet. The 0.08 kW host allowance is a measured desktop overhead: a study that metered an RTX 5080 workstation at the wall beside the GPU's own reported power found the difference at 68-80 W across its ordinary ML and HPC workloads (arXiv 2606.19262), and the allowance takes the top of that range.
Same 96 GB and the same 1,792 GB/s as the full-power Workstation Edition at half the board power: a power-capped bin for slim chassis, not a cut-down memory system. It sells for the same price, too: $16,000.00 on NVIDIA's own store. Compute throughput at 300 W is lower, and this catalog holds no measurement of by how much. Carries no NVLink: the datasheet lists PCIe 5.0 x16 as the whole of its system interface, and the Server Edition's product brief states 'NVIDIA NVLink: Not supported' outright for the same silicon. NVLink left this product line two generations ago — the RTX A6000 had a bridge for two cards at 112 GB/s, the RTX 6000 Ada that replaced it did not — so a second card in the same box talks over the host bus at 128 GB/s, not over a fabric.
0.3 kW is the card alone, the Max-Q's 300 W board power. The 0.08 kW host allowance is a measured desktop overhead: a study that metered an RTX 5080 workstation at the wall beside the GPU's own reported power found the difference at 68-80 W across its ordinary ML and HPC workloads (arXiv 2606.19262), and the allowance takes the top of that range.
Four is the count NVIDIA itself names: the Max-Q datasheet says "Scale up to four RTX PRO 6000 Max-Q GPUs", and the 600 W Workstation Edition datasheet makes no such claim, which is why this build is the 300 W part. Memory and bandwidth are four times the card’s own published figures, and since 2026-09-25 so is the price: four cards at NVIDIA's own $16,000.00, where it was four times a range of retailer quotes, $30,692 to $35,600. The grade is `assumption` for the 1.25 host multiplier, the one figure in this entry this repository chose. Power is the four cards' board power plus a host allowance, which its own record cites. The H100 page is cited for one thing only, and a review was right that nothing said so: NVIDIA prints "NVLink: 900GB/s | PCIe Gen5: 128GB/s" in one spec table, which fixes both the figure and the convention it is on, since the 900 is the bidirectional total this catalog already stores for H100 SXM. The A6000 page dates the loss of NVLink from the line.
1.2 kW is the four cards alone, four times the Max-Q's 300 W board power. The 0.08 kW host allowance is a measured desktop overhead: a study that metered an RTX 5080 workstation at the wall beside the GPU's own reported power found the difference at 68-80 W across its ordinary ML and HPC workloads (arXiv 2606.19262), and the allowance takes the top of that range. That machine held one card; a host for four carries a larger CPU and more fans, so 0.08 kW is a floor for this build, and no measurement of a four-card host was found.
The rack card carries the same 96 GB GDDR7 as the two workstation editions but publishes 1,597 GB/s against their 1,792 GB/s — a lower memory clock, stated on both NVIDIA's own page and an OEM product guide, so it is not a typo on one page. Decode is bound by that figure, which makes the server part the slowest of the three despite the datacenter label. Carries no NVLink: the datasheet lists PCIe 5.0 x16 as the whole of its system interface, and the Server Edition's product brief states 'NVIDIA NVLink: Not supported' outright for the same silicon. NVLink left this product line two generations ago — the RTX A6000 had a bridge for two cards at 112 GB/s, the RTX 6000 Ada that replaced it did not — so a second card in the same box talks over the host bus at 128 GB/s, not over a fabric.
0.6 kW is the card alone, its 600 W maximum. The card runs in eight-slot servers, so the 0.575 kW host allowance is one eighth of the host share of the eight-GPU system whose accelerators are closest to 600 W. NVIDIA's DGX H200 datasheet gives 10.2 kW maximum system power for its standard configuration, and its eight H200s are rated "Up to 700W" each on NVIDIA's H200 page: 5.6 kW of accelerators, which leaves 4.6 kW for the CPUs, memory, NVSwitch chips, networking, fans and power-supply losses.
Previous generation and still sold new. It is here because "the 96 GB one or the 48 GB one" describes two generations rather than two configurations of one card: 96 GB is Blackwell, 48 GB is Ada, and the Ada part also has barely half the memory bandwidth.
0.3 kW is the card alone, its 300 W maximum. The 0.08 kW host allowance is a measured desktop overhead: a study that metered an RTX 5080 workstation at the wall beside the GPU's own reported power found the difference at 68-80 W across its ordinary ML and HPC workloads (arXiv 2606.19262), and the allowance takes the top of that range.
Same GB202 die and the same 1,792 GB/s as the RTX PRO 6000, with a third of the memory. That pairing is why it is listed: on a consumer card it is capacity that runs out first, not bandwidth. NVIDIA's own page rendered neither price nor bandwidth in this session's fetch, so both are corroborated rather than read off the vendor.
0.575 kW is the card alone, its 575 W total graphics power. The 0.08 kW host allowance is a measured desktop overhead: a study that metered an RTX 5080 workstation at the wall beside the GPU's own reported power found the difference at 68-80 W across its ordinary ML and HPC workloads (arXiv 2606.19262), and the allowance takes the top of that range.
128 GB of unified LPDDR5x buys capacity no discrete desktop card offers, at 273 GB/s — about a seventh of an RTX PRO 6000. Decode is memory-bound, so this box fits models it cannot serve quickly, and the catalog holds no throughput measurement for it. Power is a measured full-load draw, recorded with its source in its own record.
ServeTheHome measured the whole box at "just under 200W" with its CPU and GPU both loaded, and never reached 240 W; many LLM inference workloads ran at 60-90 W. 0.2 kW is that full-load figure, the basis the other systems here are charged at. It replaces 0.24 kW, the rating of the 240 W USB-PD supply, which bounds the draw rather than measuring it. The GB10 chip's own TDP is 140 W.
32 GB GDDR6 at 640 GB/s and 300 W, from AMD's own product page. The price is press reporting of the launch figure, not a line on that page.
0.3 kW is the card alone, its 300 W total board power. The 0.08 kW host allowance is a measured desktop overhead: a study that metered an RTX 5080 workstation at the wall beside the GPU's own reported power found the difference at 68-80 W across its ordinary ML and HPC workloads (arXiv 2606.19262), and the allowance takes the top of that range.
128 GB of LPDDR5x at 8,000 MT/s and 256 GB/s, from AMD's specification page, which names the GPU as a Radeon 8060S with 40 RDNA 3.5 compute units. The memory is unified, and AMD's developer software sets how much of it the GPU may use; AMD's pages give no ceiling for that on this processor, so the entry records all 128 GB, as the other unified-memory entries do. AMD promised pre-orders for June 2026 on 2026-05-20 and said on 2026-07-06 that the systems "are now available at Micro Center", the date recorded here. A Halo with the Ryzen AI Max+ PRO 495 and 192 GB is announced with no price or date and is not this entry. No throughput profile here was measured on this system, so it carries no curated rate and needs a manual one.
AMD publishes no measured draw for this box. 0.15 kW is its own planning figure: the cost model on its Ryzen AI Halo page assumes a "150W sustained" draw for the whole system, which it calls the "nightmare case". The specification page gives a TDP of 120 W, which is the processor's alone, so it is not the figure charged. Graded an assumption because AMD states it as one, not as a measurement or a specification.
Apple states 153 GB/s for the M6 with 16 GB and 170 GB/s once it is configured to 24 or 32 GB, and 32 GB is the most an M6 takes. Orders opened on 2026-08-25 and deliveries began on 2026-09-22. No throughput profile here was measured on Metal, so this system carries no curated rate and needs a manual one.
Apple measures Mac mini at the wall, power-supply losses included, while "a compute-intensive test application" maximizes processor usage. Its one M6 figure, 70 W, was measured with 16 GB and a 256 GB SSD. It publishes none for this 32 GB, 1 TB build, whose memory also runs faster, 170 GB/s against 153, so 0.07 kW carries that measurement across and errs low.
Apple states 307 GB/s for the M5 Pro and lists no other figure for its 20-core GPU bin, and 64 GB is the most an M5 Pro takes. Orders opened on 2026-08-25 and deliveries began on 2026-09-22. No throughput profile here was measured on Metal, so this system carries no curated rate and needs a manual one.
Apple measures Mac mini at the wall, power-supply losses included, while "a compute-intensive test application" maximizes processor usage. 145 W is its measurement of an M5 Pro with 64 GB, taken with an 8 TB SSD. The page does not say which of the two GPU bins it measured, so the confidence is medium rather than high. The figure sits under the 155 W that Apple's specifications give as every Mac mini's maximum continuous power.
Apple publishes two M5 Max bandwidths, 460 GB/s for the 32-core GPU and 614 GB/s for the 40-core, and sells more than 36 GB only with the 40-core, so a 128 GB M5 Max is always the faster bin. Every throughput profile in this catalog was measured on a CUDA or ROCm serving stack and none of them transfers to Metal, so this system carries no curated rate and needs a manual one.
Apple measures Mac Studio at the wall, power-supply losses included, while "a compute-intensive test application" maximizes processor usage: 200 W for the M5 Max with the 32-core GPU and 36 GB, and 385 W for the M5 Ultra with the 80-core GPU and 512 GB. It publishes no figure for this 40-core, 128 GB configuration, so 0.2 kW carries the 32-core measurement across; more GPU cores draw more at full load, so it errs low. It replaces 0.48 kW, the supply's maximum continuous rating, which bounds the draw rather than measuring it.
Apple sells 256 GB and 512 GB only with the 80-core GPU, and states 1.2 TB/s for both M5 Ultra bins. Orders opened on 2026-08-25 and deliveries began on 2026-09-22. As on the M5 Max, no throughput profile here was measured on Metal, so this system carries no curated rate and needs a manual one.
Apple measures Mac Studio at the wall, power-supply losses included, while "a compute-intensive test application" maximizes processor usage. Its one M5 Ultra figure, 385 W, was measured on the same 80-core GPU with 512 GB and a 16 TB SSD. It publishes none for this 256 GB, 1 TB build, so 0.385 kW carries that measurement across; half the memory and a smaller SSD draw less, so it errs high, against ownership.
The largest unified memory Apple sells, on the same 80-core GPU and 1.2 TB/s as the 256 GB build; only the memory differs. It is an entry of its own because the machine is on sale and this configuration is not: one entry marked `announced` would say something false about the 256 GB build. As on every Mac Studio here, no throughput profile was measured on Metal, so it needs a manual rate.
Apple measures Mac Studio at the wall, power-supply losses included, while "a compute-intensive test application" maximizes processor usage. 385 W is its measurement of this configuration, the 80-core GPU with 512 GB, taken with a 16 TB SSD.
“Late October” is read as the month's last third, the 21st to the 31st. Apple's store still said “512GB memory option for M5 Ultra coming late October” on 2026-09-25.
Parameter counts, context, quantization scheme and licence read from the published model card, config.json and LICENSE. The 1,560.86 GB checkpoint size is the safetensors index's own metadata.total_size, 1,560,860,324,864 bytes, rather than a figure quoted from a secondary write-up. Moonshot's technical blog, not the model card, carries the deployment floor: "we recommend deploying Kimi K3 on supernode configurations with 64 or more accelerators." [CORRECTED 2026-09-13: this note gave the size as 1,560.94 GB, recomputed from the repository's shard byte sizes, after the model's own notes had moved to the index.]
Derived from config.json: 24 Gated-MLA layers cache kv_lora_rank 512 + qk_rope_head_dim 64 = 576 elements per token, and 69 Kimi Delta Attention layers hold a fixed state of linear_attn_config num_heads 96 x head_dim 128 x 128 = 1,572,864 elements at fp32, plus three short-convolution windows of 96 x 128 x (short_conv_kernel_size 4 - 1) = 36,864 elements each at bf16: 6,512,640 bytes per layer, 449.37 MB per sequence. VALIDATED against SGLang's published figures: 24 x 576 x 2 = 27,648 bytes per token at bf16, or 27.0 KiB, against their stated "MLA KV block per token: 27 KB"; and one rank of eight holds 69 x (12 x 128 x 128 x 4 + 3 x 12 x 128 x 3 x 2) = 56,171,520 bytes of recurrent state against their stated "KDA state per request: 54 MB under TP=8". That is the sum SGLang itself computes — its ratio calculator for this model multiplies out exactly these terms — and it rounds to 54 in binary megabytes, 53.57 MiB, where decimal megabytes would read 56.17. Note that mla_use_nope does not remove the rope channel from the cache: the rotation is skipped, the 64 dimensions are still stored. The MLA latent is a single KV head, so plain tensor parallelism replicates it on every rank; decode context parallelism shards it, which is why SGLang's 48-concurrent-session result used DCP8. [CORRECTED 2026-08-21: this profile defaulted to a one-byte element, alone among the twenty-five profiles here, on the reading that SGLang serves K3 with an fp8 cache. It does not. SGLang's own published block size — one MLA KV block per token of about 27 KB across all 24 MLA layers — is the bf16 figure, since 24 x 576 x 2 = 27,648 bytes and one byte would give 13.5 KiB. Their cookbook offers --kv-cache-dtype fp8_e4m3 as a lever that halves KV bytes per token, which is only meaningful from a two-byte baseline, and the checkpoint's own quantization_config carries kv_cache_scheme: null, so nothing in the release prescribes fp8. The default model on the front page was therefore showing half its cache: 3.24 GB where it should read 6.00 GB, and 133 concurrent sequences where 71 fit. The fp8 path is real and stays available as a control; it is not the default.] [CORRECTED 2026-09-13: the state was 69 x 1,720,320 elements at fp32, 474.81 MB per sequence. That counted each short-convolution window as the whole kernel of four inputs and stored it at fp32 beside the state, but a window keeps short_conv_kernel_size - 1 = 3 inputs, and both runtimes hold it at the activation width while the state stays at float32. vLLM e52be1a6 sizes the window conv_kernel_size - 1 + num_spec in kda_state_shape (model_executor/layers/mamba/mamba_utils.py:298) and takes it at the model dtype in kda_state_dtype (:133); SGLang fa663e72 sizes it conv_kernel_size - 1 in KimiLinearStateShape.create (srt/configs/mamba_utils.py:324) at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348). The validation sentence used to put 69 x 12 x 128 x 128 x 4 = 54.26 MB against the same published 54 and conclude that "their figure covers the recurrent state alone, so the convolutions here add about 9% on top of it". It matched for the wrong reason: 54.26 is the fp32 state alone in decimal megabytes, SGLang counts window and state together (mamba_cache_per_req, srt/configs/mamba_utils.py:116), and at their true size the windows add 3.52% to the state.] The split by head is exact while the tensor-parallel size divides the 96 heads: 1, 4 and 8 do and 72 does not, and both runtimes' state shapes refuse a size that does not. vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state.
models/moonshotai/Kimi-K3.yaml (updated 2026-09-10) marks B200, B300, GB200 and GB300 verified, and its default configuration there adds --kv-cache-dtype fp8 through its Blackwell override. Its H200 and MI355X configurations leave the cache at the model's width, and its general fp8 tip names only the Blackwell profile's attention backend, so vLLM's recipe leaves Hopper and CDNA 4 unknown. CDNA 4 has a record of its own, read from SGLang.
SGLang's Kimi K3 cookbook gives MI350X and MI355X one recipe, and it stores the cache at fp8: the page's command generator emits --kv-cache-dtype fp8_e4m3 for both, and its platform table lists an fp8 kvcache for them. SGLang pull request #39029 ran that recipe on 8× MI350X (gfx950) as the generator emitted it, with --context-length 16384 added, and all six of its runs finished without a scheduler exception. AMD's MI350X page gives its GPU architecture as CDNA4, which is the architecture this catalog records for the MI355X system, and the MI355X runs the same recipe. Graded vendor-reported because the engine's project published those runs, not because a cell is marked verified: the page marks every MI350X and MI355X cell Final Verification In Progress, which it says means the recipe runs while its serving round on the final weights and current code is still open. Medium confidence because the runs used the recipe as it stood before pull request #39066 moved it to --dcp-size 8 and AITER attention on a newer image on 2026-09-22, still at fp8_e4m3, and no run of that form is published. This records that the cache runs, not that it keeps the model's quality: the page calls its precision levers, the fp8_e4m3 cache among them, accuracy-gated, and says accuracy has not been re-measured on any cell.
config.json linear_attn_config: 69 KDA layers and 24 full Gated-MLA layers, 93 total.
config.json: num_experts 896, num_experts_per_token 16, num_shared_experts 2, moe_intermediate_size 3072.
2 x num_key_value_heads 8 x head_dim 128 = 2,048 elements per token per layer across 126 layers: 516.10 MB per 1,000 tokens at bf16. VALIDATED against a published figure — Table 1 of DeepSeek's ISCA 2025 hardware paper tabulates Llama-3.1-405B at 516.096 KB per token at bf16, which this reproduces exactly. Kept in the catalog after retirement precisely because it is the baseline every newer architecture is measured against: the cheapest model here costs a hundredth of this per token of context.
126 layers, 8 KV heads, head dimension 128, no sliding window and no compression. Read from Table 3 of Meta's own Llama 3 paper rather than from config.json: Meta gates the model repository, so the config file cannot be cited as something a reader can check. The previously cited source here was the llama-models GitHub repository, which carries licence and policy text and no architecture table at all — the numbers were right and the citation did not support them. Five independently uploaded derivatives of the same checkpoint agree with the paper on every field.
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 7 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
2 x num_key_value_heads 8 x head_dim 128 = 2,048 elements per token per layer. config.json no_rope_layers marks every fourth layer as global NoPE — 12 of 48 — and attention_chunk_size 8192 caps the other 36. Past 8,192 tokens the marginal cost is 49.15 MB per 1,000 tokens against 196.61 MB inside the chunk, but the chunked layers saturate at 1.21 GB per sequence, the largest fixed term in this catalog.
config.json: 40 query heads over 8 KV heads at head dimension 128, across 48 layers. no_rope_layers marks 12 of those 48 as global NoPE full-attention and the remaining 36 as chunked at attention_chunk_size 8,192; transformers reads a 1 in that array as a RoPE chunked layer and a 0 as a NoPE global one. [CORRECTED 2026-08-21: this note previously said the chunk size and the chunked-layer set could not be read because Meta gates the repository. That was true of the gated file and false of the entry — the kvCache block beside it had already sized the cache from an ungated mirror two weeks earlier, so one half of this model's record contradicted the other. Re-verified against two independently uploaded mirrors of the same checkpoint, which agree on every field.]
config.json: num_local_experts 128, num_experts_per_tok 1, one shared expert per MoE layer. interleave_moe_layer_step 2, so only 24 of the 48 layers are MoE at all.
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 10 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
84 shards summing to 416,755,504,016 bytes.
Model card: "23 x (3 x (Gated DeltaNet -> MoE) -> 1 x (Gated Attention -> MoE))" over 92 layers. The 23 growing layers cache 2 x num_key_value_heads 4 x head_dim 256 = 2,048 elements per token — 94.21 MB per 1,000 tokens. The 69 linear layers hold linear_num_value_heads 128 x linear_value_head_dim 128 x linear_key_head_dim 128 = 2,097,152 state elements at fp32 plus a convolution window over q and k at linear_num_key_heads 16 and v at the 128 value heads, (2 x 16 x 128 + 128 x 128) x (linear_conv_kernel_dim 4 - 1) = 61,440 elements at bf16, about 587 MB per sequence: the largest constant term in this catalog, and a real constraint on how many sequences a deployment can hold. The recurrent-state layout is the one both runtimes allocate for Gated DeltaNet: vLLM e52be1a6's gated_delta_net_state_shape (model_executor/layers/mamba/mamba_utils.py:274) and SGLang fa663e72's Mamba2StateShape.create with the key heads as its groups (srt/configs/qwen3_next.py:300). [CORRECTED 2026-09-13: the convolution state was 81,920 elements at fp32 and the constant 601.42 MB per sequence. The window keeps linear_conv_kernel_dim - 1 = 3 inputs, not 4, and both runtimes store it at the model's bfloat16 while the state follows mamba_ssm_dtype, float32: vLLM takes the window at the model dtype and the state from mamba_ssm_cache_dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98), which model_executor/models/config.py sets from this checkpoint's mamba_ssm_dtype; SGLang takes the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348), and the state from the same field.] The split by head is exact while the tensor-parallel size divides both the 16 key heads and the 128 value heads: 1, 4 and 8 do and 72 does not. vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state. [UPGRADED 2026-09-13: confidence medium to high. The layout was inferred from the Gated DeltaNet formulation; it is now read from the code both runtimes allocate it with, and every field it uses from this checkpoint's config.json.]
config.json layer_types over 92 layers: 69 linear_attention, 23 full_attention, full_attention_interval 4.
config.json uses non-standard key names: num_experts 512 (not n_routed_experts), num_experts_per_tok 10, and the shared expert appears only as shared_expert_intermediate_size 2048.
safetensors index metadata.total_size 2,496,066,252,544 bytes.
safetensors index metadata.total_size 1,444,420,107,432 bytes across 569,081 tensors, read through /resolve/ because /raw/ returns a git-LFS pointer for this repository. Graded an official claim rather than a measurement: NVIDIA publishes the artefact and its size, not an accuracy evaluation against the bf16 baseline, so nothing here says what the quantization cost in quality.
Only 16 layers grow with context: 2 x num_key_value_heads 4 x head_dim 256 = 2,048 elements per token per layer, or 64 KiB per token, or 65.54 MB per 1,000 tokens across the model at bf16 — a twentieth of what a same-sized model with attention on every layer would hold. The 48 linear layers hold a fixed state: linear_num_value_heads 48 x linear_key_head_dim 128 x linear_value_head_dim 128 = 786,432 elements at fp32, plus a convolution window over q and k at linear_num_key_heads 16 and v at the 48 value heads, (2 x 16 x 128 + 48 x 128) x (linear_conv_kernel_dim 4 - 1) = 30,720 elements at bf16: 3,207,168 bytes per layer, 153.94 MB per sequence. The per-token slope is read straight from the config. The recurrent-state layout is the one both runtimes allocate for Gated DeltaNet — vLLM e52be1a6's gated_delta_net_state_shape (model_executor/layers/mamba/mamba_utils.py:274) and SGLang fa663e72's Mamba2StateShape.create with the key heads as its groups (srt/configs/qwen3_next.py:300) — and no vendor number exists to validate it against, unlike Kimi K3's. An error there shifts the constant term and not the slope. [CORRECTED 2026-09-13: the convolution term was 40,960 elements at fp32, 827,392 elements per layer and 158.86 MB per sequence. The window keeps linear_conv_kernel_dim - 1 = 3 inputs, not 4, and both runtimes store it at the model's bfloat16 while the state follows mamba_ssm_dtype, float32: vLLM takes the window at the model dtype and the state from mamba_ssm_cache_dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98), which model_executor/models/config.py sets from this checkpoint's mamba_ssm_dtype; SGLang takes the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348), and the state from the same field.] The split by head is exact while the tensor-parallel size divides both the 16 key heads and the 48 value heads: 1, 4 and 8 do and 72 does not. vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state. [UPGRADED 2026-09-13: confidence medium to high. The layout was inferred from the Gated DeltaNet formulation; it is now read from the code both runtimes allocate it with, and every field it uses from this checkpoint's config.json.]
config.json layer_types: 48 linear_attention and 16 full_attention, full_attention_interval 4. The model card describes it as "16 × (3 × (Gated DeltaNet → FFN) → 1 × (Gated Attention → FFN))".
Summed from the repository file listing: 30,866,866,928 bytes across 64 layer shards plus mtp.safetensors and outside.safetensors. No metadata.total_size is published for this repository.
Size read from the repository file listing. The accuracy figure is the quantizer’s own published measurement, reproducible from the logs, calibration corpus and reference logits they publish alongside it. [DOWNGRADED 2026-10-07: graded measured until this date. The KL divergence and top-1 figures are AtomicChat's own run of its own build, which the taxonomy grades vendor-reported, and AtomicChat's repository is the primary source for the build it published.]
Size from the safetensors index's metadata.total_size: 21,921,428,072 bytes, the sum of the headers of all 3 files. The accuracy figures are NVIDIA's own evaluation of its build against the bf16 release; the repository publishes no evaluation logs or reference logits alongside them. [DOWNGRADED 2026-10-07: graded measured until this date. These accuracy figures are NVIDIA's own run of its own build, which the taxonomy grades vendor-reported, and NVIDIA's repository is the primary source for the build it published.]
Size read from the repository file listing. The accuracy figure is the quantizer’s own published measurement, reproducible from the logs, calibration corpus and reference logits they publish alongside it. [DOWNGRADED 2026-10-07: graded measured until this date. The KL divergence and top-1 figures are AtomicChat's own run of its own build, which the taxonomy grades vendor-reported, and AtomicChat's repository is the primary source for the build it published.]
Size read from the repository file listing. The accuracy figure is the quantizer’s own published measurement, reproducible from the logs, calibration corpus and reference logits they publish alongside it. [DOWNGRADED 2026-10-07: graded measured until this date. The KL divergence and top-1 figures are AtomicChat's own run of its own build, which the taxonomy grades vendor-reported, and AtomicChat's repository is the primary source for the build it published.]
2 x num_key_value_heads 2 x head_dim 128 = 512 elements per token per layer. Two KV heads is aggressive even by grouped-query standards, and it is the main reason this model costs 13.31 MB per 1,000 tokens at long context — a twenty-fifth of Hunyuan Hy3. The cache stops dividing beyond two accelerators. CROSS-CHECKED 2026-08-27 against Sebastian Raschka's independently published BF16 cache-per-token figures: counting every layer as growing, before the window cap, this geometry gives 52.00 KiB per token against his 52 KiB. An exact match.
Model card: "Attention pattern | [Local, Local, Local, Global] repeating"; "RoPE (theta = 500,000), local layers only". config.json layer_types puts global attention at every fourth layer — 13 global and 39 sliding of 52 — with sliding_window 2048 and only 2 KV heads.
6 attention layers at 512 elements per token: 6.14 MB per 1,000 tokens. State-space layers hold mamba_num_heads 64 x mamba_head_dim 64 x ssm_state_size 128 = 524,288 elements at fp32 plus a convolution window of (64 x 64 + 2 x n_groups 8 x 128) x (conv_kernel 4 - 1) = 18,432 elements at bf16, about 49 MB per sequence. That is the layout both runtimes allocate for Mamba-2: vLLM e52be1a6's mamba2_state_shape (model_executor/layers/mamba/mamba_utils.py:195) and SGLang fa663e72's Mamba2StateShape.create (srt/configs/mamba_utils.py:195). [CORRECTED 2026-09-13: the convolution state was 24,576 elements at fp32, about 50 MB per sequence. A Mamba-2 window keeps conv_kernel - 1 = 3 inputs, and both runtimes store it at the model's bfloat16 while the state stays at float32: vLLM takes the window at the model dtype and the state from mamba_ssm_cache_dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98), which model_executor/models/config.py sets from this checkpoint's own mamba_ssm_cache_dtype; SGLang takes the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348), and the state at its float32 default.] The split by head is exact while the tensor-parallel size divides both the 8 groups and the 64 heads: 1, 4 and 8 do and 72 does not, and vLLM's MambaMixer2 asserts both (model_executor/layers/mamba/mamba_mixer2.py:305 and :309, the second waived only for a single group). vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state.
models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml (updated 2026-09-22) marks H100 verified, and its default configuration there sets --kv-cache-dtype fp8 in its default variant. That configuration is the recipe's default checkpoint, NVIDIA's NVFP4 build (nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4), which the recipe describes as "with FP8 KV cache — the default deployment target".
models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml (updated 2026-09-22) marks DGX Station (GB300) verified, and its default configuration there sets --kv-cache-dtype fp8 in its default variant. That configuration is the recipe's default checkpoint, NVIDIA's NVFP4 build (nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4), which the recipe describes as "with FP8 KV cache — the default deployment target".
models/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16.yaml (updated 2026-09-22) marks DGX Spark (GB10) and RTX PRO 6000 verified, and its default configuration there sets --kv-cache-dtype fp8 in its default variant. That configuration is the recipe's default checkpoint, NVIDIA's NVFP4 build (nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4), which the recipe describes as "with FP8 KV cache — the default deployment target".
config.json layers_block_type over 52 entries: 23 mamba, 23 moe and 6 attention.
2 x num_key_value_heads 8 x head_dim 128 = 2,048 elements per token per layer everywhere; only the window differs. Past 4,096 tokens the marginal cost is 81.92 MB per 1,000 tokens, with 46.7 MB of fixed window state.
Model card: "1 x Global (NoPE) / 1 x 4096 SWA / 19 x [3 x 128 SWA + 1 x Global] Blocks", and the sliding_windows array matches exactly — 20 layers at 0 (full attention), one at 4,096 and 57 at 128.
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 20 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
Attention layers cache 2 x num_key_value_heads 2 x head_dim 128 = 512 elements per token, so the model adds only 12.29 MB per 1,000 tokens. The state-space layers hold mamba_num_heads 256 x mamba_head_dim 64 x ssm_state_size 128 = 2,097,152 elements at fp32 plus a convolution window of (256 x 64 + 2 x n_groups 8 x 128) x (conv_kernel 4 - 1) = 55,296 elements at bf16, giving 8,499,200 bytes per layer — about 408 MB per sequence before a single token of context. That is the layout both runtimes allocate for Mamba-2: vLLM e52be1a6's mamba2_state_shape (model_executor/layers/mamba/mamba_utils.py:195) and SGLang fa663e72's Mamba2StateShape.create (srt/configs/mamba_utils.py:195). [CORRECTED 2026-09-13: the convolution state was 73,728 elements at fp32, 2,170,880 elements per layer and about 417 MB per sequence. A Mamba-2 window keeps conv_kernel - 1 = 3 inputs, and both runtimes store it at the model's bfloat16 while the state stays at float32: vLLM takes the window at the model dtype and the state from mamba_ssm_cache_dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98), which model_executor/models/config.py sets from this checkpoint's own mamba_ssm_cache_dtype; SGLang takes the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348), and the state at its float32 default.] The split by head is exact while the tensor-parallel size divides both the 8 groups and the 256 heads: 1, 4 and 8 do and 72 does not, and vLLM's MambaMixer2 asserts both (model_executor/layers/mamba/mamba_mixer2.py:305 and :309, the second waived only for a single group). vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state.
models/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16.yaml (updated 2026-09-10) marks B200 verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments.
config.json layers_block_type over 108 entries: 48 mamba, 48 moe and 12 attention. NVIDIA warns of the consequence in its own report — below about 64K tokens the 32-bit Mamba state is larger than the FP8 KV cache, so the fixed term dominates rather than the growing one.
config.json: 2 (key and value) x num_key_value_heads 4 x head_dim 128 = 1,024 elements per token per layer, across 60 layers — 120 KiB of key and value per token at bf16, which is more per token than either MLA model here despite being a much smaller model. That is the trade grouped-query attention makes. Sparse attention selects the top 16 blocks of 128 to *read*, which reduces arithmetic and not storage, so it does not shrink this figure. THE 60 LAYERS ARE TWO GROUPS BECAUSE OF WHAT A STEP READS, NOT WHAT THEY STORE: sparse_attention_config selects on layers 3 to 59, the 57 sparse_attention_freq marks sparse, and layers 0 to 2 attend densely to every position. A selection caps every layer of a group it reaches, so the three dense layers sit in a group of their own marked readsEveryEntry, and both bandwidth searches charge them every entry and the 57 sparse layers 2,048 positions. Both groups store the same 1,024 elements per token, so no memory figure moves. [CORRECTED 2026-10-07: the 60 layers were one group, which capped the three dense layers at 2,048 positions as well, an understatement growing with context: at 100,000 tokens on DGX B300 the least a step reads of one request was charged 12,176,916,480 bytes, and is 13,380,550,656.] Only 4 KV heads means the cache stops dividing beyond 4 accelerators and replicates thereafter. THE INDEX BRANCH KEEPS A KEY-ONLY CACHE OF ITS OWN, and both engines allocate it as a pool of its own: one 128-element key per token on each of the 57 layers sparse_attention_freq marks sparse, shared by the sparse_num_index_heads 4 index query heads and with no values beside it, since sparse_disable_index_value is set on every one of those layers. vLLM e52be1a6 budgets it as one key-only vector per token with a single head (models/minimax_m3/common/indexer.py:145-153) on every sparse layer (models/minimax_m3/nvidia/model.py:100, 698-702), at indexer_kv_dtype, whose auto default for this model is bf16 (model.py:499-503, config/attention.py:83-86, 135-139). SGLang fa663e72 keeps it in a K-only pool with one head per sparse layer (srt/mem_cache/memory_pool.py:5233-5235, 5421-5432), at the model's bf16 unless its fp8 attention-GEMM mode is on (srt/mem_cache/kv_cache_configurator.py:1802-1814), and sizes it as 57 x 128 x 2 bytes per token (srt/model_executor/pool_configurator.py:360-397). Neither engine lets --kv-cache-dtype alone move it. So the group is 57 x 128 elements at 2 bytes whatever Cache precision says, replicated, since one key head has nothing to split across ranks: 14,592 bytes per token, 11.9% on top of the key and value. Per-token slope 60 x 1,024 x 2 + 57 x 128 x 2 = 137,472 bytes, 134.25 KiB per token or 137.47 MB per 1,000 tokens at bf16. [CORRECTED 2026-10-01: the index branch's keys were declared and left out, so the profile was partial, on the reading that the release does not say whether they are stored once per token or once per index head, a term between 11.9% and 47.5% of the key and value. Both engines' pools settle it at the lower end, and GLM-5.3-Flash and Qwen3.8-Flash-Next had priced the same kind of cache since 2026-08-27.]
config.json: 64 query heads over 4 KV heads at head dimension 128, across 60 layers. sparse_attention_config selects the top 16 blocks of 128 on layers 3 to 59; layers 0 to 2 attend densely. Sparse *selection* does not shrink the cache — the entries have to exist to be selectable — so this is sized as an ordinary grouped-query cache, which is the honest reading and the conservative one.
config.json: num_local_experts 128, num_experts_per_tok 4, n_shared_experts 1.
safetensors index metadata.total_size 451,543,283,200 bytes.
Size summed from the repository file listing: 250,103,762,320 bytes across 88 safetensors shards, header bytes included, because the index declares no metadata.total_size. The accuracy figures are NVIDIA's own evaluation of its build, and unlike the AtomicChat entries the repository publishes no evaluation logs, calibration corpus or reference logits alongside them. [CORRECTED 2026-09-13: this note said they were reproducible from logs, a calibration corpus and reference logits published alongside them, the sentence the AtomicChat GGUF entries carry, where it is true.] [DOWNGRADED 2026-10-07: graded measured until this date. These accuracy figures are NVIDIA's own run of its own build, which the taxonomy grades vendor-reported, and NVIDIA's repository is the primary source for the build it published.]
2 (key and value) x num_key_value_heads 8 x head_dim 128 = 2,048 elements per token per layer, across all 80 layers: 327.68 MB per 1,000 tokens at bf16. That is 26.7 times what Nemotron 3 Ultra costs at nearly twice the parameter count, and the clearest illustration in this catalog that long-context cost is set by attention design rather than by parameter count. [CORRECTED 2026-08-21: this read 'forty times a Mamba hybrid of twice the size', which welded two different comparisons together — 40x is the ratio against Nemotron 3 Super, a model 0.4 times this size, not 2 times it.]
Model card: "Attention Heads | 64 (GQA, 8 KV heads, head dim 128)". No sliding window, no latent compression, no linear layers.
Parameter counts and licence from the model card; the exact release day was not published, so the first of the month is recorded and the confidence lowered accordingly.
2 x num_key_value_heads 8 x head_dim 128 = 2,048 elements per token per layer throughout; the sliding layers stop at 4,096 tokens and contribute a fixed 402.7 MB per sequence thereafter. Past the window the marginal cost is 32.77 MB per 1,000 tokens against 131.07 MB inside it — a fourfold flattening.
Model card: "The attention layers interleave sliding-window attention layers with Rotational Positional Embeddings and global attention layers without positional embeddings in a 3:1 ratio". config.json layer_types confirms 24 sliding and 8 full over 32 layers, sliding_window 4096.
text_config: num_experts 128, num_experts_per_tok 8, num_shared_experts 4, expert_selection_fn sigmoid, shared_expert_combination_strategy average, first_k_dense_replace 0 - so every layer routes, with no dense prefix.
8 attention layers at 2 x 2 x 128 = 512 elements per token give 8.19 MB per 1,000 tokens. Each state-space layer holds mamba_num_heads 128 x mamba_head_dim 64 x ssm_state_size 128 = 1,048,576 elements at fp32 plus a convolution window of (128 x 64 + 2 x n_groups 8 x 128) x (conv_kernel 4 - 1) = 30,720 elements at bf16, about 170 MB per sequence in total. That is the layout both runtimes allocate for Mamba-2: vLLM e52be1a6's mamba2_state_shape (model_executor/layers/mamba/mamba_utils.py:195) and SGLang fa663e72's Mamba2StateShape.create (srt/configs/mamba_utils.py:195). [CORRECTED 2026-09-13: the convolution state was 40,960 elements at fp32, about 174 MB per sequence in total. A Mamba-2 window keeps conv_kernel - 1 = 3 inputs, and both runtimes store it at the model's bfloat16 while the state stays at float32: vLLM takes the window at the model dtype and the state from mamba_ssm_cache_dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98), which model_executor/models/config.py sets from this checkpoint's own mamba_ssm_cache_dtype; SGLang takes the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348), and the state at its float32 default.] The split by head is exact while the tensor-parallel size divides both the 8 groups and the 128 heads: 1, 4 and 8 do and 72 does not, and vLLM's MambaMixer2 asserts both (model_executor/layers/mamba/mamba_mixer2.py:305 and :309, the second waived only for a single group). vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state.
models/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16.yaml (updated 2026-09-10) marks H100 and H200 verified, and its default configuration there stores the cache at fp8 through the default variant's --kv-cache-dtype fp8, which the recipe describes as "BF16 weights (FP8 KV cache recommended)".
models/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16.yaml (updated 2026-09-10) marks B200 and DGX Station (GB300) verified, and its default configuration there stores the cache at fp8 through the default variant's --kv-cache-dtype fp8, which the recipe describes as "BF16 weights (FP8 KV cache recommended)".
models/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16.yaml (updated 2026-09-10) marks DGX Spark (GB10) and RTX PRO 6000 verified, and its default configuration there stores the cache at fp8 through the default variant's --kv-cache-dtype fp8, which the recipe describes as "BF16 weights (FP8 KV cache recommended)".
config.json hybrid_override_pattern resolves to 40 mamba, 40 moe and 8 attention layers of 88.
params.json: kv_lora_rank 512 + qk_rope_head_dim 64 = 576 cached elements per token per layer across all 61 layers, or 68.6 KiB per token at bf16. As with every MLA model here the latent is a single KV head, so ordinary tensor parallelism replicates it on every rank while the weights divide.
params.json: kv_lora_rank 512, qk_rope_head_dim 64, 61 layers. The n_kv_heads field reads 128 and is not the cache width — applying the grouped-query formula to it would overstate the cache by more than a hundredfold.
params.json: num_experts 128, num_experts_per_tok 4, num_shared_experts 1.
consolidated.safetensors.index.json metadata.total_size 1,351,982,353,920 bytes.
consolidated.safetensors.index.json metadata.total_size 403,125,497,052 bytes.
Only 24 of the 42 layers store a cache at all: num_kv_shared_layers is 18, and those layers reuse an earlier layer's keys and values instead of keeping their own. Which 18 comes from the reference implementation config.json names: Gemma4ForConditionalGeneration sets first_kv_shared_layer_idx to num_hidden_layers minus num_kv_shared_layers, 42 - 18 = 24, and every layer from 24 upward reads stored keys and values rather than projecting its own. So the storing layers are 0 through 23, and config.json's layer_types array says what each of them is — five sliding_attention then one full_attention, repeating — which puts full attention at 5, 11, 17 and 23. That is 4 global at 2 x 2 x global_head_dim 512 = 2,048 elements per token and 20 sliding at 2 x 2 x head_dim 256 = 1,024, capped at sliding_window 512: 16.38 MB per 1,000 tokens with a 21 MB fixed term. EVERY LAYER STILL READS ONE: from layer 24 each layer reads the last storing layer of its own type, 23 for full attention and 22 for sliding (transformers 93ebf6b1 Gemma4Attention store_full_length_kv and shared_kv_states; vLLM e52be1a6 kv_sharing_target_layer_name; SGLang fa663e72 kv_shared_layer_index), so 7 layers read the 4 global caches and 35 read the 20 sliding windows. The groups record those as readingLayers, which multiply what a step reads and not what memory holds. [CORRECTED 2026-09-13: both bandwidth searches charged each stored cache once.] [UPGRADED 2026-09-11 from an assumption. The previous note read "ASSUMPTION: which 18 layers share is not documented for Gemma 4, so the split between global and sliding among the storing layers follows the Gemma 3n convention that the top layers are the sharers". The convention was the right guess and no figure changes, but it was never a guess: both halves are stated, one in the config and one in the class the config names, and neither had been read. The attention record beside this one had already recorded the 5:1 layer pattern, so the entry was calling a split an assumption in one field and quoting its evidence in another.]
config.json: 5 sliding : 1 full over 42 layers, sliding_window 512, num_kv_shared_layers 18. The shared-layer mechanism means those layers materialise no cache of their own. The cache itself is sized in the kvCache block, not here. [CORRECTED 2026-09-11: this note gave the reason for that as "the global layers publish no KV head count (num_global_key_value_heads is null)". A null there is not an absent figure. The reference implementation gives every full_attention layer a head_dim override of global_head_dim 512 and adds a num_key_value_heads override only when the global field is set, so null leaves those layers on the top-level count of 2 — which is what the kvCache block had already sized them at, 2 x 2 x 512. Confidence rises from medium to high with that resolved.] [CORRECTED 2026-08-29: the second source was gemma-4-qat-hf, which resolves to google/gemma-4-31B-it-qat-w4a16-ct — a quantized build of a different model, and one that says nothing about layer types either way. The config.json quoted here is E4B's own, so the citation is now the repository it was read from.]
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 9 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
Single-file checkpoint of 11,513,861,828 bytes, re-verified against the HuggingFace API on 2026-08-16 after an internal consistency check flagged the ratio as an outlier. The bf16 reference was confirmed at the same time: 15,992,595,884 bytes over 7,996,156,490 bf16 parameters.
config.json layer_types repeats 5 sliding to 1 full over 60 layers. Sliding layers cache 2 x num_key_value_heads 16 x head_dim 256 = 8,192 elements per token but stop at sliding_window 1,024, so past the window they contribute a fixed 838.9 MB per sequence rather than a slope. Global layers take a per-layer override: transformers builds per_layer_config with head_dim = global_head_dim 512 for every full_attention layer, and attention_k_eq_v additionally swaps num_key_value_heads for num_global_key_value_heads 4. They therefore cache 2 x 4 x 512 = 4,096 elements per token, or 81.92 MB per 1,000 tokens past the window against 901.12 MB inside it. [CORRECTED 2026-08-21: this was recorded as 2,048 on the reading that attention_k_eq_v makes keys and values one stored tensor. It does not. In modeling_gemma4.py the flag only drops v_proj, after which value_states starts as the key projection's output and then diverges — k_norm plus RoPE on one, a scale-free v_norm on the other — and both are passed to past_key_values.update(). It saves a weight matrix and a matmul, not cache. Two vLLM issue threads reporting Gemma 4 page-size math independently quote 512 x 2 for the global layers. The figure was understated by exactly a factor of two.] CROSS-CHECKED 2026-08-27 against Sebastian Raschka's independently published BF16 cache-per-token figures, which agree exactly with this catalog on Muse Glimmer 30B and Qwen3.6 27B and disagree here: counting every layer as growing, before the window cap, this geometry gives 880.00 KiB per token against his 840 KiB. The whole 40 KiB is the ten global layers. 840.0 is what the ten global layers give at 2,048 elements per token instead of 4,096, which is half this entry's current figure. [CORRECTED 2026-08-28: this sentence used to attribute that halving to substituting the top-level head_dim 256 for global_head_dim 512, and to add that the same substitution is what this entry carried until the 2026-08-21 correction above. The second half is false and the first is unknowable. Two independent misreadings land on 2,048 — 4 heads × 256 with the K-and-V factor of two, or 4 × 512 without it — and the entry made the SECOND one: before a606d95 the note read “Global layers set attention_k_eq_v, meaning keys and values are one unified tensor, so they cache num_global_key_value_heads 4 × global_head_dim 512 = 2,048”, which reads the per-layer override correctly and drops the factor of two. Which of the two produced the published 840 is not decidable from the number, so this entry no longer claims to know.] config.json for google/gemma-4-31b-it declares head_dim 256 and global_head_dim 512 side by side with num_global_key_value_heads 4, and transformers applies the 512 to every full_attention layer. Two independent readers making the same substitution is the argument for stating the override explicitly rather than assuming a model has one head dimension.
config.json layer_types: a 5 sliding : 1 full pattern repeating over 60 layers, sliding_window 1024. Local and global layers have different geometry — 16 KV heads at head dimension 256 locally against 4 at 512 globally — so they must be sized separately.
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 19 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
Single-file checkpoint of 23,265,352,448 bytes.
16 growing layers at 2 x num_key_value_heads 4 x head_dim 256 = 2,048 elements per token: 65.54 MB per 1,000 tokens. Linear layers hold linear_num_value_heads 48 x 128 x 128 = 786,432 elements at fp32 plus a convolution window of (2 x linear_num_key_heads 16 x 128 + 48 x 128) x (linear_conv_kernel_dim 4 - 1) = 30,720 elements at bf16, 153.94 MB per sequence. The recurrent-state layout is the one both runtimes allocate for Gated DeltaNet — vLLM e52be1a6's gated_delta_net_state_shape (model_executor/layers/mamba/mamba_utils.py:274) and SGLang fa663e72's Mamba2StateShape.create with the key heads as its groups (srt/configs/qwen3_next.py:300) — rather than a figure published for this model; it shifts the constant term, not the slope. CROSS-CHECKED 2026-08-27 against Sebastian Raschka's independently published BF16 cache-per-token figures: this geometry gives 64.00 KiB per token against his 64 KiB. [CORRECTED 2026-08-27: that sentence used to describe the basis as “counting every layer as growing, before the window cap”, which is boilerplate from the Muse Glimmer and Gemma 4 notes and is wrong here twice over. No group on this model carries a window, and 48 of its 64 layers are Gated DeltaNet with no per-token term at all.] The 64.00 is the 16 full-attention layers alone, at 2 × 4 KV heads × 256 = 2,048 elements per token per layer, so on this entry the modelled figure and the uncapped one are the same number. An exact match. [CORRECTED 2026-08-29: every claim on this model cited qwen36-github, github.com/QwenLM/Qwen3.6, which GitHub now answers with a 301 to QwenLM/Qwen3.8 because the repository was renamed for the next generation. A renamed repository keeps no copy of what it used to hold, so the citation did not break loudly — it silently served a different generation's documentation to anyone checking this entry's arithmetic. The source record is removed and the eight provenance blocks it backed now cite the model's own Hugging Face repository, whose config.json was re-read today and agrees with every figure here: 64 layers split 16 full_attention and 48 linear_attention, num_key_value_heads 4 at head_dim 256, linear_num_value_heads 48 at key and value head dim 128, linear_conv_kernel_dim 4.] [CORRECTED 2026-09-13: the convolution state was 40,960 elements at fp32 and the constant 158.86 MB per sequence. The window keeps linear_conv_kernel_dim - 1 = 3 inputs, not 4, and both runtimes store it at the model's bfloat16 while the state follows mamba_ssm_dtype, float32: vLLM takes the window at the model dtype and the state from mamba_ssm_cache_dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98), which model_executor/models/config.py sets from this checkpoint's mamba_ssm_dtype; SGLang takes the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348), and the state from the same field.] The split by head is exact while the tensor-parallel size divides both the 16 key heads and the 48 value heads: 1, 4 and 8 do and 72 does not. vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state.
Architecturally identical to Qwen3.8-27B — same model type and every attention field matches: 64 layers with 48 linear and 16 full, 4 KV heads at head dimension 256.
10 growing layers at 2 x 2 x 256 = 1,024 elements per token: 20.48 MB per 1,000 tokens. Linear layers hold linear_num_value_heads 32 x 128 x 128 = 524,288 elements at fp32 plus a convolution window of (2 x linear_num_key_heads 16 x 128 + 32 x 128) x (linear_conv_kernel_dim 4 - 1) = 24,576 elements at bf16, about 64 MB per sequence. The convolution term is the layout both runtimes allocate for Gated DeltaNet — vLLM e52be1a6's gated_delta_net_state_shape (model_executor/layers/mamba/mamba_utils.py:274) and SGLang fa663e72's Mamba2StateShape.create with the key heads as its groups (srt/configs/qwen3_next.py:300) — rather than a figure published for this model. [CORRECTED 2026-08-29: every claim on this model cited qwen36-github, github.com/QwenLM/Qwen3.6, which GitHub now answers with a 301 to QwenLM/Qwen3.8 because the repository was renamed for the next generation. A renamed repository keeps no copy of what it used to hold, so the citation did not break loudly — it silently served a different generation's documentation to anyone checking this entry's arithmetic. The source record is removed and the eight provenance blocks it backed now cite the model's own Hugging Face repository, whose config.json was re-read today and agrees with every figure here: 40 layers split 10 full_attention and 30 linear_attention, num_key_value_heads 2 at head_dim 256, linear_num_value_heads 32 at key and value head dim 128.] [CORRECTED 2026-09-13: the convolution state was 32,768 elements at fp32, about 67 MB per sequence. The window keeps linear_conv_kernel_dim - 1 = 3 inputs, not 4, and both runtimes store it at the model's bfloat16 while the state follows mamba_ssm_dtype, float32: vLLM takes the window at the model dtype and the state from mamba_ssm_cache_dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98), which model_executor/models/config.py sets from this checkpoint's mamba_ssm_dtype; SGLang takes the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348), and the state from the same field.] The split by head is exact while the tensor-parallel size divides both the 16 key heads and the 32 value heads: 1, 4 and 8 do and 72 does not. vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state.
config.json: 40 layers, 30 linear and 10 full, 2 KV heads at head dimension 256, linear_num_value_heads 32.
Parameter count and weight size re-read 2026-08-29 from the repository's own API dtype split and safetensors index rather than from the model's name.
15 growing layers at 2 x 2 x 256 = 1,024 elements per token: 30.72 MB per 1,000 tokens. The recurrent state is linear_num_value_heads 64 x 128 x 128 = 1,048,576 elements per layer at fp32, beside a convolution window of (2 x linear_num_key_heads 16 x 128 + 64 x 128) x (linear_conv_kernel_dim 4 - 1) = 36,864 elements at bf16: 4,268,032 bytes per layer, about 192.06 MB per sequence. That is the layout both runtimes allocate for Gated DeltaNet: vLLM e52be1a6's gated_delta_net_state_shape (model_executor/layers/mamba/mamba_utils.py:274) and SGLang fa663e72's Mamba2StateShape.create with the key heads as its groups (srt/configs/qwen3_next.py:300). [CORRECTED 2026-08-21: this note previously excluded the short-convolution state on the grounds that "the key-head count it depends on is not published for this model". It is: config.json carries linear_num_key_heads 16 alongside linear_num_value_heads 64, linear_key_head_dim 128 and linear_conv_kernel_dim 4. Applying the same formula that reproduces every sibling entry exactly — (2 x 16 x 128 + 64 x 128) x 4 = 49,152 — the constant term rises from 188.74 MB to 197.59 MB, 4.7% higher. The slope is unaffected.] [CORRECTED 2026-09-13: the formula the correction of 2026-08-21 applied, (2 x 16 x 128 + 64 x 128) x 4, counted the window as the whole kernel of four inputs and priced it at fp32, and so did every sibling entry it reproduced; the constant was 1,097,728 elements per layer at fp32, 197.59 MB per sequence. The window keeps linear_conv_kernel_dim - 1 = 3 inputs, not 4, and both runtimes store it at the model's bfloat16 while the state follows mamba_ssm_dtype, float32: vLLM takes the window at the model dtype and the state from mamba_ssm_cache_dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98), which model_executor/models/config.py sets from this checkpoint's mamba_ssm_dtype; SGLang takes the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16 (srt/environ.py:1348), and the state from the same field.] The split by head is exact while the tensor-parallel size divides both the 16 key heads and the 64 value heads: 1, 4 and 8 do and 72 does not. vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state.
Model card: "15 * (3 * (Gated DeltaNet -> MoE) -> 1 * (Gated Attention -> MoE))" over 60 layers, 2 KV heads at head dimension 256, linear_num_value_heads 64.
text_config: num_experts 512, num_experts_per_tok 10, moe_intermediate_size 1024, shared_expert_intermediate_size 1024 over 60 layers at hidden_size 4096.
CONFLICT preserved: DeepSeek labels these checkpoints "FP4 + FP8 Mixed" while defaultWeightBits stores a single number. [CORRECTED 2026-09-11: defaultWeightBits read 8 while nativeFormat.bits read 4, so the entry contradicted itself. The 8 read as a conservative capacity assumption and was not one — defaultWeightBits is consulted only when no checkpoint size is published, which is never the case here, so it reached no arithmetic at all and would have doubled the estimate the first time a DeepSeek entry arrived without a published size.] It is now 4, which is also what the artefact measures: 0.588 bytes per parameter for Flash and 0.541 for Pro, against a range of 1.003 to 1.143 across the 13 eight-bit builds this catalog has measured. A schema rule now rejects the disagreement.
Read from DeepSeek's own reference implementation (inference/model.py in the release repository), which settles three things config.json alone does not. (1) ONE ENTRY IS 512 ELEMENTS, NOT 576. Attention sets nope_head_dim = head_dim - rope_head_dim = 512 - 64 = 448 and rotates kv[..., -64:], so RoPE occupies the last 64 of the 512 rather than adding to them; the cache buffer is registered as [batch, size, head_dim] and a single wkv projection fills it, so there is no separate K and V either. (2) EVERY LAYER ALSO HOLDS A 128-TOKEN SLIDING WINDOW: kv_cache_size = window_size + max_seq_len // compress_ratio, so 43 x 128 x 512 elements sit under the whole model as a fixed 5.64 MB per sequence. (3) A LAYER AT RATIO 0 HAS NO COMPRESSED ENTRIES AT ALL — the first two layers are the window and nothing else, so they contribute no slope. compress_ratios in the 0731 checkpoint gives 2 layers at 0, 21 at 4 and 20 at 128 over the 43 modelled layers, so the compressed slope is 21 x 128 + 20 x 4 = 2,768 elements per token, 5,536 bytes at bf16. [CORRECTED 2026-08-21: this entry read 576 elements per entry with no window term and an unbounded slope on the two ratio-0 layers, giving 8.53 MB per 1,000 tokens — 1.54x the figure for the same cache. The 576 was DeepSeek V3's concatenated kv_lora_rank + qk_rope_head_dim carried across to an architecture that dropped both fields.] THE SPARSE INDEXER KEEPS KEYS OF ITS OWN, on the 21 ratio-4 layers and nowhere else: Attention builds an Indexer only where compress_ratio is 4 (inference/model.py:474-477), and the Indexer keeps one 128-element key per compressed position, shared by all 64 index heads: it registers kv_cache as [max_batch_size, max_seq_len // compress_ratio, index_head_dim] and scores every head's query against that one key (395, 405, 426). The reference stores the key at bf16, and neither engine does. vLLM e52be1a6 builds the indexer only on ratio-4 layers (models/deepseek_v4/attention.py:293-294), stores one key per compress_ratio tokens (898-904) as 128 fp8 values and one fp32 scale, 132 bytes (996-999), and replicates it on every rank (959); SGLang fa663e72 stores the same 132 bytes (srt/mem_cache/deepseek_v4_memory_pool.py:34-38) and sizes a quarter of them per token on each ratio-4 layer (srt/model_executor/pool_configurator.py:1103). Neither width follows --kv-cache-dtype, and both engines offer a 68-byte fp4 key only behind an option of its own. So the group is 21 x 32 elements per token at 1.03125 bytes each, 693 bytes per token whatever Cache precision says, and the slope is 6,229 bytes: 6.23 MB per 1,000 tokens at bf16. [CORRECTED 2026-09-13: this said the release does not state whether the keys are stored per head (index_n_heads 64) or shared.] [CORRECTED 2026-10-01: the indexer's keys were declared and left out, so the profile was partial, while GLM-5.3-Flash and Qwen3.8-Flash-Next priced the same kind of cache; the slope read 5.54 MB per 1,000 tokens.] AT FP8 AN ENTRY IS 584 BYTES, NOT 512. Each engine stores a 512-element entry, compressed or in the window, in one of three layouts depending on its attention backend: bf16, 1,024 bytes; a per-tensor fp8 row, 512 bytes, which vLLM's FlashInfer backends and SGLang's trtllm backend keep (vllm/models/deepseek_v4/attention.py:125-129; srt/mem_cache/deepseek_v4_memory_pool.py:188-212); and DeepSeek's fp8_ds_mla layout, 448 fp8 values, the 64 rotary elements at bf16 and 8 bytes of scales and padding, 584 bytes. vLLM e52be1a6 uses that one unless a backend overrides it, and then even at --kv-cache-dtype auto (attention.py:94-123, 152), for the compressed entries (862-864) and the window alike, whose cache it builds at the same dtype (341-348; v1/attention/backends/mla/sparse_swa.py:107-124). SGLang fa663e72's DeepSeekV4SingleKVPool asserts the same 584 bytes (deepseek_v4_memory_pool.py:117-135), and the window pool and both compressed pools are built from one class, that one unless the unified Triton pool, bf16 throughout (515-568), or the trtllm pool is chosen (641-644, 706-757, 935-952). So in each engine the window is stored in the same layout as the compressed entries. Recording the larger of what the engines store gives 1,024 bytes at bf16, as before, and 584 at fp8, so every latent group carries fp8BytesPerElement 1.140625, while the indexer keys keep their 132 bytes at both settings. At fp8 the slope is then 2,768 x 1.140625 + 693 = 3,850.25 bytes per token and the window 3,214,336 bytes per sequence. Both engines round a page of 584-byte entries up to a multiple of 576 bytes, which is page padding and, like any runtime padding, not counted here. [CORRECTED 2026-10-07: fp8 charged every entry 512 bytes, the per-tensor row, so the slope read 3,461 bytes per token and the window 2,818,048 bytes per sequence, 12.3% below the engines 584-byte layout on every entry and 10.1% below it on the whole slope.] The window is modelled as a ramp capped at 128 tokens rather than as a flat constant, so a sequence shorter than the window is not charged for the whole of it — the same way Gemma 4's window is expressed, which is what makes the two entries comparable on their constant terms.
models/deepseek-ai/DeepSeek-V4-Flash.yaml (updated 2026-09-23) marks H200 verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments.
models/deepseek-ai/DeepSeek-V4-Flash.yaml (updated 2026-09-23) marks B200, GB200, B300, GB300 and DGX Station (GB300) verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments.
models/deepseek-ai/DeepSeek-V4-Flash.yaml (updated 2026-09-23) marks DGX Spark (GB10) and eight RTX PRO 6000 verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments.
models/deepseek-ai/DeepSeek-V4-Flash.yaml (updated 2026-09-23) marks MI300X and MI325X verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments.
models/deepseek-ai/DeepSeek-V4-Flash.yaml (updated 2026-09-23) marks MI355X verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments.
Technical report section 4.2.1: 43 layers, first two pure sliding-window, the rest CSA and HCA "in an interleaved manner". CSA compression rate 4 with top-k 512; HCA compression rate 128; sliding window 128; 64 query heads at head dimension 512. The report never states how many layers are of each kind, and gives no per-token cache figure, so the cache is not sized here. It does say the uncompressed sliding-window entries "exist in every layer" and run "approximately 8 times larger than the compressed CSA and HCA KV entries" — which means the window, not the compressed part, dominates. WHAT A STEP READS is in the release's own inference/model.py, not the report. index_topk counts compressed entries: the Indexer keeps topk(min(index_topk, end_pos // ratio)) of the compressed index keys (line 433), so a ratio-4 layer reads at most 512 entries of four tokens each, 2,048 tokens' worth, which is that group's tokensPerEntry. A ratio-128 layer builds no indexer (lines 474-477), and get_compress_topk_idxs hands it every compressed position (lines 275-282, called at 519), so that group reads every entry. vLLM e52be1a6 and SGLang fa663e72 select the same way: index_topk entries on ratio-4 layers and every compressed entry on ratio-128 layers. [CORRECTED 2026-09-13: selectedEntries was selectedTokens, and the bandwidth lower bound charged the ratio-4 layers 512 tokens rather than 512 entries and capped the ratio-128 layers at 512 tokens as well.]
config.json: n_routed_experts 256, num_experts_per_tok 6, n_shared_experts 1.
Size verified to the byte against the repository listing. Accuracy is the quantizer’s own published KL-divergence and top-1 measurement, with per-file logs. [DOWNGRADED 2026-10-07: graded measured until this date. The KL divergence and top-1 figures are AtomicChat's own run of its own build, which the taxonomy grades vendor-reported, and AtomicChat's repository is the primary source for the build it published.]
CONFLICT preserved on the active parameter count: 49B is the preview figure and DeepSeek has not republished it for the 0813 build, while one third-party redistribution says 48B. Total parameters and checkpoint size are solid; the active count is inherited. [CORRECTED 2026-09-11: defaultWeightBits read 8 while nativeFormat.bits read 4, so the entry contradicted itself. The 8 read as a conservative capacity assumption and was not one — defaultWeightBits is consulted only when no checkpoint size is published, which is never the case here, so it reached no arithmetic at all and would have doubled the estimate the first time a DeepSeek entry arrived without a published size.] It is now 4, which is also what the artefact measures: 0.588 bytes per parameter for Flash and 0.541 for Pro, against a range of 1.003 to 1.143 across the 13 eight-bit builds this catalog has measured. A schema rule now rejects the disagreement.
Same derivation as V4 Flash, from the same reference implementation: one entry is head_dim 512 elements with RoPE occupying its last 64, not 512 + 64, and every layer carries a 128-token sliding window on top of its compressed entries. compress_ratios in the 0813 checkpoint gives 30 layers at ratio 4 and 31 at 128 across the 61 modelled layers — no ratio-0 layers here, unlike Flash — so the compressed slope is 30 x 128 + 31 x 4 = 3,964 elements per token, 7,928 bytes at bf16, over a fixed 8.00 MB per sequence of window. [CORRECTED 2026-08-21: recorded as 8.92 MB per 1,000 tokens with no window term, on the same 576-element reading that Flash carried.] The sparse indexer keeps keys of its own on the 30 ratio-4 layers, stored as V4 Flash's are and cited there: one 128-element key per compressed position, shared by all 64 index heads in the reference code (inference/model.py:395, 405, 426, byte-identical to V4 Flash's), and held by both engines as 128 fp8 values and one fp32 scale, 132 bytes per four tokens, whatever --kv-cache-dtype says. So the group is 30 x 32 elements per token at 1.03125 bytes each, 990 bytes per token, replicated, and the slope is 8,918 bytes: 8.92 MB per 1,000 tokens at bf16. That is within a byte of the figure the 576-element reading gave, by coincidence: that reading overstated every entry by an eighth, and the indexer adds almost exactly an eighth. [CORRECTED 2026-09-13: this said the release does not state its storage layout.] [CORRECTED 2026-10-01: the indexer's keys were declared and left out, so the profile was partial; the slope read 7.93 MB per 1,000 tokens.] At fp8 an entry, compressed or in the window, is 584 bytes and not 512, stored as V4 Flash's are and cited there: DeepSeek's fp8_ds_mla layout of 448 fp8 values, the 64 rotary elements at bf16 and 8 bytes of scales and padding, which vLLM e52be1a6 uses unless a backend overrides it and SGLang fa663e72's DeepSeekV4SingleKVPool stores, where the per-tensor fp8 row of vLLM's FlashInfer backends and SGLang's trtllm backend is 512. Recording the larger gives every latent group fp8BytesPerElement 1.140625, so at fp8 the slope is 3,964 x 1.140625 + 990 = 5,511.44 bytes per token and the window 4,559,872 bytes per sequence; bf16 stays at 1,024 bytes an entry. [CORRECTED 2026-10-07: fp8 charged every entry 512 bytes, so the slope read 4,954 bytes per token and the window 3,997,696 bytes per sequence.] The window is modelled as a ramp capped at 128 tokens rather than as a flat constant, so a sequence shorter than the window is not charged for the whole of it — the same way Gemma 4's window is expressed, which is what makes the two entries comparable on their constant terms.
models/deepseek-ai/DeepSeek-V4-Pro.yaml (updated 2026-09-14) marks H200 verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments.
models/deepseek-ai/DeepSeek-V4-Pro.yaml (updated 2026-09-14) marks B200, GB200, B300 and GB300 verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments.
models/deepseek-ai/DeepSeek-V4-Pro.yaml (updated 2026-09-14) marks MI355X verified, and its default configuration there sets --kv-cache-dtype fp8 in its base arguments. It marks MI300X and MI325X unsupported for the model as a whole, which is not a statement about the cache, so CDNA 3 stays unknown.
Technical report section 4.2.1: 61 layers, first two HCA, the rest CSA and HCA interleaved. CSA compression rate 4 with top-k 1,024; HCA compression rate 128; sliding window 128; 128 query heads at head dimension 512. The layer split is not published, so the cache is not sized here. WHAT A STEP READS is in the release's inference/model.py, which is byte-identical to V4 Flash's, where the lines are cited: index_topk counts compressed entries, so a ratio-4 layer reads at most 1,024 entries of four tokens each, 4,096 tokens' worth, and a ratio-128 layer builds no indexer and reads every compressed entry. [CORRECTED 2026-09-13: selectedEntries was selectedTokens, and the bandwidth lower bound charged every layer 1,024 tokens.]
config.json: n_routed_experts 384, num_experts_per_tok 6, n_shared_experts 1.
Key and value head widths differ here, so the usual doubling does not apply: it is kv_heads x (head_dim + v_head_dim). Global layers give 4 x (192 + 128) = 1,280 elements per token; sliding layers give 8 x 320 = 2,560, capped at 128 tokens for a fixed 25.6 MB per sequence. Long-context marginal cost is 23.04 MB per 1,000 tokens.
Model card: "interleaving Sliding Window Attention (SWA) and Global Attention (GA) with a 5:1 ratio and 128 sliding window. This reduces KV-cache storage by nearly 6x." The hybrid_layer_pattern in config.json actually resolves to 9 global and 39 sliding of 48, which is 4.3:1 rather than 5:1. CONFLICT: the card states 8 KV heads for global and 4 for sliding, and the config states the reverse; the config is used here, and the card itself carries a notice that config.json was corrected after release.
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 25 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
kv_heads 8 x (head_dim 192 + v_head_dim 128) = 2,560 elements per token on both layer kinds; only the window differs. 51.20 MB per 1,000 tokens at long context, with 39.32 MB of fixed window state.
config.json hybrid_layer_pattern resolves to 10 global and 60 sliding layers of 70, sliding_window 128, with 8 KV heads and split key/value head dimensions of 192 and 128 throughout.
config.json: num_key_value_heads 8 on the global layers and swa_num_key_value_heads 8 on the sliding ones, head_dim and swa_head_dim 192, v_head_dim and swa_v_head_dim 128. Key and value widths differ, so the usual doubling does not apply: every layer holds 8 x (192 + 128) = 2,560 elements per token. hybrid_layer_pattern resolves to 10 global and 60 sliding layers of 70, sliding_window 128, and the report’s Table 1 gives the same. 51.20 MB per 1,000 tokens at long context, with 39.32 MB per sequence of fixed window state. The three MTP layers in model_mtp.safetensors, the report’s MTP-3 configuration, and the five-layer DFlash drafter in dflash/ are speculative-decoding drafts, and neither one’s cache is counted.
config.json hybrid_layer_pattern resolves to 10 global and 60 sliding layers of 70, the first of them global with a dense FFN, and sliding_window is 128; the model card and the report’s Table 1 agree.
config.json: n_routed_experts 384, num_experts_per_tok 8, n_shared_experts null, and moe_layer_freq makes every layer but the first a mixture of experts. The report: “Both the SWA block and the GA block utilize a sparse MoE FFN without shared experts.”
config.json: num_key_value_heads 4 on the global layers and swa_num_key_value_heads 8 on the sliding ones, head_dim and swa_head_dim 192, v_head_dim and swa_v_head_dim 128. Key and value widths differ, so the usual doubling does not apply: global layers hold 4 x (192 + 128) = 1,280 elements per token and sliding layers 8 x 320 = 2,560. hybrid_layer_pattern resolves to 9 global and 39 sliding layers of 48, sliding_window 128. The model card and the report’s Table 1 give the same head counts, where MiMo V2.5’s card had them the other way round. 23.04 MB per 1,000 tokens at long context, with 25.56 MB per sequence of fixed window state. The three MTP layers in model_mtp.safetensors, the report’s MTP-3 configuration, and the five-layer DFlash drafter in dflash/ are speculative-decoding drafts, and neither one’s cache is counted.
config.json hybrid_layer_pattern resolves to 9 global and 39 sliding layers of 48, the first of them global with a dense FFN, and sliding_window is 128; the model card and the report’s Table 1 agree.
config.json: n_routed_experts 256, num_experts_per_tok 8, n_shared_experts null, and moe_layer_freq makes every layer but the first a mixture of experts. The report: “Both the SWA block and the GA block utilize a sparse MoE FFN without shared experts.”
Read from the artefact rather than inherited, since 2026-08-29. The active-parameter count is the one figure still not from Z.ai: they have never published one for either version, and 40B is Artificial Analysis's.
config.json: 78 layers, kv_lora_rank 512 plus qk_rope_head_dim 64 for a 576-element MLA latent, num_nextn_predict_layers 1. Previously inherited from GLM-5.2 and now read from GLM-5.3's own config, which agrees with it. [DOWNGRADED 2026-08-29: this was the only one of the catalog's 33 KV geometries graded official-claim, and it is now estimated like the other 32. Z.ai publishes no per-token cache figure for GLM-5.3. What the config publishes is kv_lora_rank and qk_rope_head_dim; adding them, multiplying by 78 layers and calling the product a cache size is this repository's arithmetic, and the GLM-5.2 entry, which performs that same arithmetic on those same two fields, has always said estimated.] The DSA lightning indexer keeps its own key cache, priced here as on GLM-5.2. Of the 78 layers modelled above, the 21 that indexer_types marks full keep one, and the 57 marked shared keep none and reuse the top-k of the full layer before them. vLLM e52be1a6 serves it as DeepSeek V3.2 (models/deepseek_v32/__init__.py:21), builds an indexer only on the full layers or an MTP layer, reading them off index_topk_freq 4 and index_skip_topk_offset 3 (models/deepseek_v32/attention.py:166-190), and holds each key as 128 fp8 values and one fp32 scale, 132 bytes per token, in a uint8 pool that --kv-cache-dtype does not reach (91-100). SGLang fa663e72 builds none on a layer dsa_layer_skips_topk marks shared (srt/configs/model_config.py:311-319, srt/models/deepseek_v2.py:1808-1841), stores the same 132 bytes per token (srt/mem_cache/index_key_cache.py:32-38) and gives a shared layer a zero-row buffer (40-43). One key head has nothing to split, so both replicate it. So the group is 21 x 128 elements at 1.03125 bytes, 2,772 bytes per token whatever Cache precision says, 3.1% on top of the latent's 87.75 KiB: 92,628 bytes, 90.46 KiB per token at bf16. SGLang drops the zero-row buffers only on a plain launch: with HiCache, hierarchical sparse attention, an external cache linker or prefill-decode disaggregation every layer allocates the key (srt/mem_cache/kv_cache_configurator.py:105-113), 10,296 bytes per token, which this entry does not price. The MTP layer (num_nextn_predict_layers 1) is not one of the 78: both runtimes build it only as the draft model for MTP speculative decoding (vLLM config/speculative.py:666-681, SGLang srt/configs/model_config.py:789-790), with a latent cache and an indexer key cache of its own, 576 x 2 + 132 = 1,284 bytes per token at bf16, which this entry does not count any more than it counts the drafting. [CORRECTED 2026-09-13: this said the number of distinct indexer key caches is between 22 and 78 and that the config does not say which, computed its floor from 21 all the same, and put the ceiling at nearly four times that. The 22 counted the MTP layer, which the 78-layer latent leaves out, and neither runtime keeps a key cache on a shared layer, so 78 is no ceiling. The declaration was here while this entry inherited GLM-5.2's geometry and went missing when the geometry was re-derived from GLM-5.3's own config, which is how one edit raised the stated confidence and dropped the caveat that qualified it.] [CORRECTED 2026-10-01: the indexer's key cache was declared and left out, so the profile was partial, and the note put the keys at bf16, 21 x 128 x 2 bytes = 5.25 KiB per token and 6.0% on top of the latent, where both engines store fp8 with a scale; the MTP layer's figure read 576 + 128 elements, 1,408 bytes per token at bf16.]
models/zai-org/GLM-5.3.yaml (updated 2026-09-18) marks H200 verified, and its default configuration there sets --kv-cache-dtype fp8 through its Hopper override, commented "H100/H200 use plain fp8 KV cache".
models/zai-org/GLM-5.3.yaml (updated 2026-09-18) marks B300 and DGX Station (GB300) verified, and its default configuration there sets --kv-cache-dtype fp8_e4m3 in its base arguments.
Inherited from GLM-5.2 on the vendor’s same-base-model statement. index_topk 2048 counts tokens, and every layer reads that many from its own latent cache: vLLM e52be1a6 builds the indexer with topk_tokens = config.index_topk (models/deepseek_v32/attention.py:65), and a shared layer builds none and attends with the indices of the full layer before it (166-190). So selectedEntries is 2,048 on all 78 layers. [CORRECTED 2026-10-01: the entry was not marked selective, so the bandwidth lower bound charged every layer its whole latent cache.]
config.json: n_routed_experts 256, num_experts_per_tok 8, n_shared_experts 1, first_k_dense_replace 3. Identical to GLM-5.2, as the vendor said it would be.
Both repositories' first commits, each titled "Initial commit 0828", landed at 2026-08-27T17:02Z (BF16) and 17:16Z (FP8), which is 2026-08-28 in Beijing, the day both titles name and the day Z.ai's promise resolved to; both are ungated. Z.ai's announcement on 2026-08-14 promised the weights “in two weeks after launch, once safety evaluation and hardening are complete”, which put the target at 2026-08-28. It landed on the 28th. [CORRECTED 2026-10-01: this note dated the FP8 repository by its last modification when it was read, 2026-08-29T09:51:13Z, which is a merged evaluation pull request, and said the weights landed on the 28th and the 29th.]
metadata.total_size 1,506,659,919,872 bytes over 59,585 tensors, byte-identical to GLM-5.2's index.
Size from the safetensors index's metadata.total_size: 464,115,789,896 bytes, the sum of the headers of all 50 files. The accuracy figures are NVIDIA's own evaluation of its build against Z.ai's FP8 release, and like the GLM-5.2 build's the repository publishes no evaluation logs or reference logits alongside them. The card puts the saving at "approximately 1.66x" against the FP8 build; the two indexes give 1.63x. [DOWNGRADED 2026-10-07: graded measured until this date. These accuracy figures are NVIDIA's own run of its own build, which the taxonomy grades vendor-reported, and NVIDIA's repository is the primary source for the build it published.]
CONFLICT preserved: two independent reads of the bf16 checkpoint gave 1,506,659,919,872 bytes (the safetensors index’s declared total) and 1,506,689,458,421 bytes (the sum of the shards on disk). The declared total is carried, because tensor bytes rather than file bytes are what has to fit in accelerator memory; the difference is 0.002%. CONFLICT preserved on the release date, 2026-06-16: Hugging Face’s activity feed records the repository as published at 2026-06-16T07:39:21Z, and Z.ai’s release notes list GLM-5.2 on 2026-06-16, but IT之家 reported on 2026-06-17 that Z.ai announced the launch and open-sourcing “today”. 2026-06-16 is carried because the weights could be downloaded that day, and the press report is dated after it.
Derived from config.json: kv_lora_rank 512 + qk_rope_head_dim 64 = 576 cached elements per token per layer, across all 78 layers, 87.75 KiB of latent per token at bf16. Two independent passes agreed on the fields. NOTE the trap: num_key_value_heads reads 64, but that is a compatibility field equal to num_attention_heads — applying the ordinary grouped-query formula to it would overstate this cache by roughly fifty times. The DSA lightning indexer keeps a key cache of its own, and only on the 21 layers indexer_types marks full; the 57 marked shared build no indexer and reuse the top-k of the full layer before them. Each key is one head of index_head_dim 128, shared by all index_n_heads 32. vLLM e52be1a6 serves it as DeepSeek V3.2 (models/deepseek_v32/__init__.py:21), builds an indexer only on the full layers or an MTP layer, reading them off index_topk_freq 4 and index_skip_topk_offset 3 (models/deepseek_v32/attention.py:166-190), and holds each key as 128 fp8 values and one fp32 scale, 132 bytes per token, in a uint8 pool that --kv-cache-dtype does not reach (91-100). SGLang fa663e72 builds none on a layer dsa_layer_skips_topk marks shared (srt/configs/model_config.py:311-319, srt/models/deepseek_v2.py:1808-1841), stores the same 132 bytes per token (srt/mem_cache/index_key_cache.py:32-38) and gives a shared layer a zero-row buffer (40-43). One key head has nothing to split, so both replicate it. So the group is 21 x 128 elements at 1.03125 bytes, 2,772 bytes per token whatever Cache precision says, 3.1% on top of the latent's 87.75 KiB: 92,628 bytes, 90.46 KiB per token at bf16. SGLang drops the zero-row buffers only on a plain launch: with HiCache, hierarchical sparse attention, an external cache linker or prefill-decode disaggregation every layer allocates the key (srt/mem_cache/kv_cache_configurator.py:105-113), 10,296 bytes per token, which this entry does not price. Nor does it price the MTP layer's own indexer, which both engines build only when that layer drafts. [CORRECTED 2026-10-01: this note left the indexer out, about 5% low it said, because how the cache shards under tensor parallelism was not published, and the profile was partial. Both engines replicate it, and the GLM-5.3 and Hunyuan Hy4-preview notes had quoted its layers and its width since 2026-09-13.]
models/zai-org/GLM-5.2.yaml (updated 2026-09-18) marks B300 and B200 verified, and its default configuration there sets --kv-cache-dtype fp8_e4m3 in its base arguments.
models/zai-org/GLM-5.2.yaml (updated 2026-09-18) marks MI300X verified, and its default configuration there sets --kv-cache-dtype fp8_e4m3 in its base arguments.
models/zai-org/GLM-5.2.yaml (updated 2026-09-18) marks MI355X verified, and its default configuration there sets --kv-cache-dtype fp8_e4m3 in its base arguments.
models/zai-org/GLM-5.2.yaml (updated 2026-09-18) sets --kv-cache-dtype fp8 for Hopper in its Hopper override, commented "H100/H200 use plain fp8 KV cache", and its guide launches the FP8 checkpoint on 8xH200 with that flag. It marks no Hopper GPU verified, so this is a published configuration rather than a reported run.
Z.ai states the consequence plainly: "Although the new GLM-5.2 architecture reduces per-token computational FLOPs, it does not proportionally reduce per-token KV-cache size." index_topk 2048 counts tokens, and every layer reads that many from its own latent cache: vLLM e52be1a6 builds the indexer with topk_tokens = config.index_topk (models/deepseek_v32/attention.py:65), and a shared layer builds none and attends with the indices of the full layer before it (166-190). So selectedEntries is 2,048 on all 78 layers. [CORRECTED 2026-10-01: the entry was not marked selective, so the bandwidth lower bound charged every layer its whole latent cache.]
config.json: n_routed_experts 256, num_experts_per_tok 8, n_shared_experts 1, first_k_dense_replace 3 (layers 0-2 are dense).
safetensors index metadata.total_size 755,617,140,416 bytes.
Size from the safetensors index's metadata.total_size: 464,795,267,072 bytes. The accuracy figures are NVIDIA's own evaluation of its build against Z.ai's GLM-5.2-FP8, and unlike the AtomicChat entries the repository publishes no evaluation logs, calibration corpus or reference logits alongside them. [CORRECTED 2026-09-13: the size was 464.82 GB, the 464,823,042,096 bytes of the repository's 47 files, which include each file's header as well as its tensors; and this note said the accuracy figures were reproducible from logs, a calibration corpus and reference logits published alongside them, the sentence the AtomicChat GGUF entries carry, where it is true.] [DOWNGRADED 2026-10-07: graded measured until this date. These accuracy figures are NVIDIA's own run of its own build, which the taxonomy grades vendor-reported, and NVIDIA's repository is the primary source for the build it published.]
Derived from text_config: num_hidden_layers 61, and every layer caches kv_lora_rank 512 + qk_rope_head_dim 64 = 576 elements per token. num_nextn_predict_layers is 0, so unlike GLM-5.2 the safetensors index carries exactly 61 layer indices and there is no extra MTP layer to account for. num_key_value_heads reads 64, which on a latent-attention model does not describe the cache at all - it is the count of query-side heads the absorbed latent serves, and reading a KV size off it would overstate this model's cache by roughly 22x. The latent is a single KV head, so plain tensor parallelism replicates it on every rank; only decode context parallelism shards it. defaultBytesPerElement is 2 because quantization_config.kv_cache_scheme is null: the release prescribes no fp8 cache, so bf16 is the reference store, which is the rule this catalog applies everywhere else. fp8 remains available as a runtime lever.
text_config: kv_lora_rank 512, q_lora_rank 1536, qk_nope_head_dim 128, qk_rope_head_dim 64, v_head_dim 128, num_attention_heads 64.
text_config: n_routed_experts 384, num_experts_per_tok 8, n_shared_experts 1.
Total parameters are the repository's own safetensors count, 321,323,031,390, rather than the rounded 320B the model card prints — the same convention the glm-5-2 entry uses. Active parameters 18B is Z.ai's own published figure, not an aggregator's. Weight size is metadata.total_size for the default FP8 repository, 328,326,771,576 bytes, corroborated independently by vLLM's recipe at about 306 GiB. Context read from config.json rather than from the docs page's "1M".
Derived from config.json and modeling_glm5_next.py. layer_types splits 45 layers into 34 linear_attention and 11 deepseek_sparse_attention, and kda_layers lists exactly the 34. The latent is kv_lora_rank 512 plus qk_rope_head_dim 0 = 512 elements per token, not GLM-5.2's 576: mla_use_nope is true and the k_rot slice is empty, so copying the sibling entry's figure would overstate this model by 12.5%. The indexer keeps its own key cache on those same 11 layers, from a single shared key head, which is why that group is replicated rather than by-kv-head: there is nothing to split across tensor-parallel ranks, so at TP=8 it costs eight times what a per-rank division would suggest. The reference implementation packs 128 key elements, 128 gate scores and one validity flag, 257 elements per token, and neither engine stores that: both keep each key at fp8 with one fp32 scale, 128 + 4 = 132 bytes whatever --kv-cache-dtype says, and hold gate scores only for the index_kpool 4 tokens of the group still open. SGLang fa663e72 stores one key per token (DSATokenToKVPool, srt/mem_cache/memory_pool.py:4791-4871, and srt/mem_cache/index_key_cache.py:15-37); vLLM e52be1a6 stores one per complete group of 4, a quarter as many (models/glm5next/nvidia/attention.py:78-157 and 278-287). Where the engines an entry cites differ, this catalog records the larger, so a fit it reports holds under either: 128 elements per token at 1.03125 bytes each, SGLang's 132 bytes per layer. Both engines also keep each request's open group raw, 4 bf16 keys and 4 bf16 gate scores per layer, 11 x 4 x 2 x 128 x 2 = 22,528 bytes per request (vLLM attention.py:159-205 and 288-298, SGLang memory_pool.py:4894-4944), under 0.02% of the per-sequence constant below, and not modelled. [CORRECTED 2026-09-26: this note read "The indexer stores 128 key elements, 128 gate scores and one validity flag = 257 elements per token per sparse-attention layer" and "index_kpool 4 pools at scoring time out of the full stored tensor and does not divide the storage", priced the group at 257 elements per token at whatever width Cache precision set, and gave the per-token slope as 11 x 512 + 11 x 257 = 8,459 elements, 16.92 MB per 1,000 tokens at bf16, 5.31x smaller than the glm-5-2 entry's.] Each linear layer holds linear_attn_config num_heads 64 x head_dim 128 x 128 = 1,048,576 state elements, which the code casts to fp32 before storing them, plus three convolution windows of 64 x 128 x (short_conv_kernel_size 4 - 1) = 24,576 elements each at bf16. Per-token slope 11 x 512 x 2 + 11 x 132 = 12,716 bytes, 12.72 MB per 1,000 tokens at bf16, and 11 x 512 + 11 x 132 = 7,084 bytes at fp8, where the indexer's 1,452 bytes stay as they are; per-sequence constant 34 x (1,048,576 x 4 + 3 x 24,576 x 2) bytes = 147.62 MB per sequence. That per-token slope is 7.07x smaller than the glm-5-2 entry's, which is the quantitative content of Z.ai's claim to cut long-context serving cost. NOT VALIDATED against a published block size: unlike Kimi K3 no per-token or per-block figure is published for this model, and vLLM's KV-pool token counts cannot be converted to bytes per token without assumptions that move the answer threefold. The fp8 cache path is real but hardware-gated: vLLM's own recipe runs this model's cache at fp8 through its Blackwell override, verified on B200 and GB200, states that Hopper must run a bf16 KV cache, and also marks MI355X and the Ascend 950PR verified, both at the model's own width and without a word on fp8. The comment on its Blackwell override, "FP8 KV cache only works on Blackwell for this model; Hopper runs BF16 KV.", is read as a refusal on Hopper, which it names, and not on CDNA 4 or the Ascend 950PR: a GPU the recipe runs at the model's own width is a recipe that did not try. [CORRECTED 2026-09-26: this note said "so the fp8 control here is reachable on Blackwell and newer only". The records refuse fp8 on Hopper alone, so the control stays open on every other architecture: on Blackwell sm_100/sm_103 its line says vLLM runs it, and elsewhere that whether an engine runs fp8 is unknown.] [CORRECTED 2026-09-13: the windows were 32,768 elements each at fp32, 1,146,880 elements per layer and 155.98 MB per sequence, and the fp32 cast was stated of all of it. It is true of the state and only of the state: modeling_glm5_next.py at transformers 415e6d2f casts last_recurrent_state to torch.float32 at line 729 before update_recurrent_state, while the convolution window goes through causal_conv1d_update and update_conv_state (lines 662-674) with no cast. Both runtimes keep that window for short_conv_kernel_size - 1 = 3 inputs at the activation width and the state at float32. vLLM e52be1a6's Glm5Next KDA layer calls kda_state_dtype with the model dtype and no SSM override (models/glm5next/nvidia/kda.py:136), so --mamba-ssm-cache-dtype does not reach this model's state there, and kda_state_shape (model_executor/layers/mamba/mamba_utils.py:298) sizes the window; SGLang fa663e72 builds KimiLinearStateShape from linear_attn_config (srt/configs/glm5_next.py:265) with the window at SGLANG_MAMBA_CONV_DTYPE, default bfloat16, and its --mamba-ssm-dtype bfloat16 halves the state.] The split by head is exact while the tensor-parallel size divides the 64 heads: 1, 4 and 8 do and 72 does not.
models/zai-org/GLM-5.3-Flash.yaml (updated 2026-09-18) marks B200 and GB200 verified, and its default configuration there adds --kv-cache-dtype fp8 through its Blackwell override.
models/zai-org/GLM-5.3-Flash.yaml (updated 2026-09-18) comments its Blackwell override "FP8 KV cache only works on Blackwell for this model; Hopper runs BF16 KV", and its guide says "Hopper does not support FP8 KV cache for this model and must run BF16 KV". It marks H100 verified with the cache at the model's width.
config.json layer_types: 34 linear_attention, 11 deepseek_sparse_attention, 45 total.
config.json: n_routed_experts 288, num_experts_per_tok 8, n_shared_experts 1, first_k_dense_replace 3, moe_intermediate_size 2048.
metadata.total_size 642,646,653,816 bytes.
Size from the safetensors index's metadata.total_size: 204,419,110,596 bytes, the sum of the headers of all 33 files. The accuracy figures are NVIDIA's own evaluation of its build against Z.ai's GLM-5.3-Flash-BF16; the repository publishes no evaluation logs or reference logits alongside them. The card's prose counts the shared experts among the quantized layers; its config excludes them and their headers are bf16, and this entry follows the artefact. [DOWNGRADED 2026-10-07: graded measured until this date. These accuracy figures are NVIDIA's own run of its own build, which the taxonomy grades vendor-reported, and NVIDIA's repository is the primary source for the build it published.]
Weight size is metadata.total_size for the default BF16 repository, 359,999,963,128 bytes — declared as a JSON float, alone in this catalog, which a strict integer parser rejects. It reconciles exactly with the dtype breakdown: 179,999,981,424 BF16 parameters x 2 plus 35 I64 x 8. Context read from config.json, which sets max_position_embeddings to 262,144; the served qwen3.8-flash API advertises 1M as a production feature added on top of this checkpoint, and the two are not the same artefact.
Derived from config.json and modeling_qwen4_exp.py. layer_types splits 48 layers into 36 linear_attention and 12 full_attention, consistent with full_attention_interval 4. Each full-attention layer caches 2 x num_key_value_heads 2 x head_dim 256 = 1,024 elements per token. The sparse indexer keeps its own key-only cache on those same layers, from a single shared key head, so it replicates under tensor parallelism rather than dividing. Both runtimes pool the keys of every indexer_compress_ratio 4 tokens into one stored row of indexer_kv_heads 1 x indexer_head_dim 128 = 128 elements, 32 elements per token, and hold the up to 4 raw keys of the group still open in a ring beside it: SGLang fa663e72 prices the per-token cost as "compressed keys only" and keeps the open group in "a per-request ring, not a per-token cache" (srt/mem_cache/qsa_kv_pool.py:38-47 and 125-128), and vLLM e52be1a6 keeps one compressed row per complete group and the open group in a circular buffer (models/qwen4_exp/common/qsa_cache.py:862-873 and 811-859). Both store the ring and the rows at bf16 whatever the main cache stores, so Cache precision does not reach this group: SGLang's index_state_dtype is torch.bfloat16 (qsa_kv_pool.py:36), and vLLM builds the ring at bf16 and gives the rows the attention config's indexer_kv_dtype, whose auto default for this model is bf16 (models/qwen4_exp/nvidia/indexer_qsa.py:152-164, config/attention.py:83-87 and 135-139). The ring is 12 x 4 x 128 x 2 = 12,288 bytes per sequence, a hundredth of a percent of the per-sequence constant below, and is not modelled; vLLM widens each ring row to 140 elements where the model runs with MRoPE, to carry three int64 positions, and rounds the ring up by whole groups of 4 to hold speculative drafts, 26,880 bytes with up to four draft tokens, and SGLang adds one 96-byte position ring per request. [CORRECTED 2026-09-26: this note read "indexer_compress_ratio 4 divides the scoring rather than the storage", priced the group at 128 elements per token at whatever width Cache precision set, and gave the per-token slope as 12 x 1,024 + 12 x 128 = 13,824 elements, 27.65 MB per 1,000 tokens at bf16.] Each linear layer holds linear_num_value_heads 48 x linear_value_head_dim 128 x linear_key_head_dim 128 = 786,432 state elements at fp32 per mamba_ssm_dtype, plus a convolution window over q and k at the key-head count and v at the value-head count, (16 x 128 x 2 + 48 x 128) x (linear_conv_kernel_dim 4 - 1) = 30,720 elements at bf16. Note the convolution convention differs from GLM's, which covers all three projections at one head count. Per-token slope 12 x 1,024 + 12 x 32 = 12,672 elements, 25.34 MB per 1,000 tokens at bf16, of which the indexer's 384 elements stay at bf16 when the rest is stored at fp8. The per-layer embedding (ple_layer_ids [2]) runs a short convolution with a window of its own: hidden_size 2560 x hc_count 4 = 10,240 channels over (ple_conv_kernel_size 4 - 1) x ngram_size 3 = 9 positions, 92,160 elements at bf16, which neither runtime divides across tensor-parallel ranks. vLLM e52be1a6 marks its MambaSpec tp_replicated (models/qwen4_exp/nvidia/model.py:828-833, ple_layer.py:175), and SGLang fa663e72 sizes it without the tensor-parallel size (srt/configs/qwen4_exp.py:96-101) in a ShortConvPool beside every state slot (srt/mem_cache/memory_pool.py:1318). Beside that SGLang keeps the last ngram_size - 1 = 2 token ids per slot, 16 bytes, which rounds to nothing here. Per-sequence constant 36 x (786,432 x 4 + 30,720 x 2) + 92,160 x 2 bytes = 115.64 MB per sequence on one accelerator. No published per-token figure exists to validate this against. [CORRECTED 2026-09-13: the window was 40,960 elements at fp32, 827,392 elements per layer and 119.14 MB per sequence, and the per-layer embedding's window was not recorded at all. The window keeps linear_conv_kernel_dim - 1 = 3 inputs, not 4, and both runtimes store it at the model's bfloat16 while the state follows mamba_ssm_dtype, float32: vLLM applies Qwen3.5's hybrid-cache contract to this architecture (Qwen4ExpForConditionalGenerationConfig, model_executor/models/config.py:858), which sets mamba_ssm_cache_dtype from that field and leaves the window at the model dtype (_mamba_state_dtype, model_executor/layers/mamba/mamba_utils.py:98); SGLang sizes it through Qwen3Next's Mamba2StateShape.create (srt/configs/qwen3_next.py:300) at SGLANG_MAMBA_CONV_DTYPE, default bfloat16.] The split by head is exact while the tensor-parallel size divides both the 16 key heads and the 48 value heads: 1, 4 and 8 do and 72 does not. vLLM's --mamba-ssm-cache-dtype bfloat16 and SGLang's --mamba-ssm-dtype bfloat16 each halve the state. Searched on 2026-09-26 for an engine that stores this cache at fp8: vLLM's recipe for the model, at 30bb6665 and unchanged on main at d71af9ad, passes no --kv-cache-dtype for any GPU, and no cell of SGLang's cookbook stores fp8, its MI350X and MI355X cells passing --kv-cache-dtype auto. Both engines' source and merged pull requests say more, one architecture at a time, and the model's fp8 records carry it, each with its sources: SGLang runs this cache at fp8 on Blackwell sm_100/sm_103 and vLLM on sm_120, and vLLM refuses it on every AMD GPU. Nothing is on record for the other architectures. [CORRECTED 2026-09-26: fp8KvSupported was false from 2026-08-27 with no source behind it, and the catalog card printed it as "No released engine ships an fp8 cache path for this model." while the calculator refused fp8 on every system. SGLang's v0.5.20 release of 2026-09-18 ships an fp8 path for this model.]
SGLang pull request #38851, merged on 2026-09-11, served this model with --kv-cache-dtype fp8_e4m3 on a GB300, compute capability 10.3, and published the results: over AIME25 and GPQA Diamond an fp8 cache scored within one standard error of bf16, with multi-token prediction and without. Pull request #38855, merged the day before, had fixed the prefill kernel that refused an fp8 cached prefix, and SGLang's v0.5.20 release of 2026-09-18 carries both. Graded vendor-reported because the engine's project merged the runs, made by a collaborator on its repository; no cookbook cell stores this model's cache at fp8, so no cell is marked verified at it. Medium confidence because the runs served RadixArk's NVFP4 build of the model rather than the bf16 release, and no fp8 run of the release is published.
vLLM pull request #55557, approved by two of the project's members and merged on 2026-09-16, let this model's sparse-attention layers store their cache at fp8 and ran it on an RTX PRO 6000 Blackwell, compute capability 12.0: served end to end at TP=1, the KV pool held 771,704 tokens at fp8 against 434,886 at bf16, and prompt log-probabilities at fp8 differed from bf16 about as much as two bf16 runs differ from each other: a standard deviation of 0.170 against 0.155 over a 150K-token prompt, 0.376 against 0.381 over 8.8K tokens of natural text and 0.015 against 0.016 over a 125K-token needle prompt. Graded vendor-reported because the engine's project merged the run; vLLM's recipe stores no GPU's cache at fp8 for this model, so none is marked verified at it. Medium confidence because no final vLLM release carries the path yet: v0.30.0 of 2026-09-22 left main at f2aad6aa of 2026-09-15, before the merge, and still refuses any cache but bf16, while the release candidate tag v0.30.1rc0 of 2026-09-23 carries it, and the recipe asks for a nightly build. The run also applied another pull request's offload on top of the branch.
vLLM loads its AMD implementation of this model on every ROCm GPU (vllm/models/qwen4_exp/__init__.py:32), and that implementation raises "Qwen4Exp QSA requires a BF16 main KV cache" for any cache width but bf16 (amd/qsa.py:113-114 and 190-191) on main on 2026-09-26, as it does in the v0.30.0 release. Pull request #55557 brought fp8 to the NVIDIA implementation only. No run of this model on a CDNA 3 GPU is published at all: the refusal comes before a kernel is chosen, so it holds on every ROCm GPU without one.
vLLM loads its AMD implementation of this model on every ROCm GPU (vllm/models/qwen4_exp/__init__.py:32), and that implementation raises "Qwen4Exp QSA requires a BF16 main KV cache" for any cache width but bf16 (amd/qsa.py:113-114 and 190-191) on main on 2026-09-26, as it does in the v0.30.0 release. Pull request #55557 brought fp8 to the NVIDIA implementation only. vLLM's recipe for the model, models/Qwen/Qwen3.8-Flash-Next.yaml, marks MI355X verified at the model's own width.
vLLM loads its AMD implementation of this model on every ROCm GPU (vllm/models/qwen4_exp/__init__.py:32), and that implementation raises "Qwen4Exp QSA requires a BF16 main KV cache" for any cache width but bf16 (amd/qsa.py:113-114 and 190-191) on main on 2026-09-26, as it does in the v0.30.0 release. Pull request #55557 brought fp8 to the NVIDIA implementation only. No run of this model on a CDNA 5 GPU is published at all: the refusal comes before a kernel is chosen, so it holds on every ROCm GPU without one.
vLLM loads its AMD implementation of this model on every ROCm GPU (vllm/models/qwen4_exp/__init__.py:32), and that implementation raises "Qwen4Exp QSA requires a BF16 main KV cache" for any cache width but bf16 (amd/qsa.py:113-114 and 190-191) on main on 2026-09-26, as it does in the v0.30.0 release. Pull request #55557 brought fp8 to the NVIDIA implementation only. No run of this model on an RDNA 4 GPU is published at all: the refusal comes before a kernel is chosen, so it holds on every ROCm GPU without one.
config.json layer_types: 36 linear_attention, 12 full_attention, 48 total, with full_attention_interval 4.
config.json: num_experts 512 — Qwen's key name, not n_routed_experts — num_experts_per_tok 10, moe_intermediate_size 640, shared_expert_intermediate_size 640.
metadata.total_size 185,502,232,570 bytes, read through /resolve/main/ because the /raw/ endpoint returns a git-LFS pointer for an index this large.
Size from the safetensors index's metadata.total_size: 132,639,846,394 bytes, the sum of the headers of all 11 files. The accuracy figures are NVIDIA's own evaluation of its build against Qwen3.8-Flash-Next-FP8; the repository publishes no evaluation logs or reference logits alongside them. [DOWNGRADED 2026-10-07: graded measured until this date. These accuracy figures are NVIDIA's own run of its own build, which the taxonomy grades vendor-reported, and NVIDIA's repository is the primary source for the build it published.]
Parameter count from the safetensors API rather than from the card, so it is the sum of the dtype split (BF16 779,930,197,312 plus F32 30,795,421) rather than a rounded headline. Active parameters are the card's 49B, which excludes the MTP layer's 0.7B and could not be derived from config.json alone.
kv_lora_rank 512 plus qk_rope_head_dim 64 gives 576 elements per token per layer across all 78 layers: 89.86 MB of latent per 1,000 tokens at bf16, against 327.68 MB for Hy3 at 295B. The same geometry as GLM-5.3, which also has 78 MLA layers at 576. The sparse-attention indexer's keys are cached too, but only where an indexer runs: vLLM e52be1a6 builds one on the 21 layers indexer_types marks "full" and on the MTP layer (num_nextn_predict_layers 1), and none on the 57 marked "shared", which reuse the top-k of the full layer before them (models/hy_v4/nvidia/attention.py:321-331). Each keeps 128 fp8 elements and one fp32 scale per token 132 bytes (145, 171, 177-182). SGLang fa663e72 serves this model as DeepSeek-style sparse attention (srt/configs/model_config.py:144-162) through the pool it gives GLM-5.3, which stores the same 132 bytes per token and a zero-row buffer on a shared layer (srt/mem_cache/index_key_cache.py:32-43). One key head has nothing to split, so both replicate it. So the group is 21 x 128 elements at 1.03125 bytes, 2,772 bytes per token whatever Cache precision says, and the slope is 92,628 bytes: 92.63 MB per 1,000 tokens at bf16. As on GLM-5.3, SGLang allocates the key on every layer under HiCache, hierarchical sparse attention or prefill-decode disaggregation, and the MTP layer's indexer exists only while that layer drafts; neither is priced. [CORRECTED 2026-09-13: this said the number of distinct indexer caches was somewhere between 22 and 78 and that the config alone does not say which. The same choice was made for GLM-5.3, which has the same DeepSeek-style indexer and the same 21-against-57 split: the latent is modelled, the indexer is declared and not guessed. GLM-5.3-Flash and Qwen3.8-Flash-Next are the two entries that do price an indexer, because their runtimes document the geometry. The roadmap carries this as an open item; the figure here is therefore a floor, not a total.] [CORRECTED 2026-08-29: that sentence read "the indexer is recorded" and pointed at an entry where it was not. GLM-5.2 declares the omission; GLM-5.3 lost the declaration when its geometry was re-derived from its own config, so this note vouched for a sibling that had gone silent. The declaration is restored on GLM-5.3 in the same change rather than the claim being trimmed to fit what was there. That sentence read "GLM-5.3-Flash is the one entry" and was wrong on the day it was written: both index groups landed on 2026-08-27 and this entry was added on 2026-08-29, so the count was two before the claim existed. Counting a population from memory rather than from the file is what produced both errors in this note.] [CORRECTED 2026-10-01: this note said the indexer's width was enough to price the cache, "which this entry still does not", and the profile was partial; the slope read 89.86 MB per 1,000 tokens.]
config.json: layer_types is 78 entries of "deepseek_sparse_attention", use_dsa true, kv_lora_rank 512 plus qk_rope_head_dim 64 for a 576-element MLA latent, index_head_dim 128, index_n_heads 32, index_topk 2048, and indexer_types splits 21 "full" against 57 "shared". index_topk counts tokens, and every layer reads that many from its own latent cache: vLLM e52be1a6 builds the Indexer with topk_tokens = index_topk (models/hy_v4/nvidia/attention.py:143) over a top-k buffer of index_topk token positions (model.py:226-233); a shared layer builds no indexer and attends with the indices of the closest preceding full layer (attention.py:54-59, 321-331); and transformers 415e6d2f writes each layer's own keys and values to the cache before choosing its top-k or taking the previous full layer's (models/hy_v4/modeling_hy_v4.py:396-397, 455-472). So selectedEntries is 2,048 on all 78 layers. [CORRECTED 2026-09-13: the entry was marked selective with no size, so the bandwidth lower bound charged these layers no cache at all.]
Estimated rather than official-claim because the repository publishes no size: metadata.total_size is absent from the index and 813.25 GB is derived from Hugging Face's per-dtype element counts (BF16 9,640,893,760, F32 30,795,421, F8_E4M3 770,289,303,552, U8 23,555,211,264) at their known widths. Confidence is medium because a second method, summing the shards on disk, disagrees by 0.06%. Format is MXFP8 rather than plain E4M3, read off quant_algo and confirmed in the tensor shapes: down_proj [256,6144,2048] against down_proj_scale [256,6144,64] is a block of 32.
29.28B is the safetensors API's single-dtype BF16 count of 29,276,770,304, not the card's rounded “30B”. Dense, so every parameter is active and no separate active count is carried.
2 (key and value) × num_key_value_heads 8 × head_dim 128 = 2,048 elements per token per layer across all 64 layers: 262.14 MB per 1,000 tokens at bf16. For the same size class that is 262.14 against 53.25 for Muse Glimmer 30B and 65.54 for Qwen3.8-27B, and 80% of what 295B Hunyuan Hy3 costs. Filling the 131,072-token window for a single sequence takes 34.4 GB of cache — more than the weights themselves at fp8.
config.json: num_hidden_layers 64, num_attention_heads 32, num_key_value_heads 8 for 4:1 grouping, and no head_dim key, so the head dimension is hidden_size / num_attention_heads = 4096 / 32 = 128. No sliding_window, no layer_types, no kv_lora_rank.
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 15 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
Parameter count from the safetensors API rather than the card, so it is the sum of the dtype split (BF16 127,486,240,128 plus F32 165,472) rather than a rounded headline. Active parameters are the card's 5.1B, which config.json alone does not give.
kv_lora_rank 512 plus qk_rope_head_dim 64 gives 576 elements per token per layer on the seven latent layers: 8.06 MB per 1,000 tokens at bf16. Three entries here are cheaper still, at 5.54, 6.14 and 7.93, and every one of them buys it the same way: by keeping most of its layers off the per-token cache. The 35 linear layers contribute no slope at all. Their state is what fla.ops.kda returns, shape [N, HV, K, V] with HV = num_attention_heads 32 and K = V = head_dim 128, so 524,288 elements, and chunk.py asserts it must be float32. Beside it sit the three short convolutions for q, k and v, each over D = 32 × 128 = 4,096 channels. The ShortConvolution cache in the reference code is [N, D, W] with W = short_conv_kernel_size 4, but the runtimes keep one input fewer: SGLang fa663e72, which this layout names, builds KimiLinearStateShape from num_attention_heads, head_dim and short_conv_kernel_size (srt/configs/bailing_hybrid.py:208) and holds 3 × 4,096 × (4 - 1) = 36,864 elements at SGLANG_MAMBA_CONV_DTYPE, default bf16. The group records the two as separate components, each at its own width: 524,288 × 4 + 36,864 × 2 = 2,170,880 bytes per layer per sequence, or 75.98 MB per sequence across the 35 layers. That constant is why this model is not automatically the cheapest choice at short context, only at long. [CORRECTED 2026-09-13: the windows were 49,152 elements, the full W = 4 of the reference cache, and the constant 76.84 MB per sequence. vLLM e52be1a6 keeps the same three inputs (kda_state_shape, model_executor/layers/mamba/mamba_utils.py:298, called from model_executor/models/bailing_moe_v3.py:618) at the model dtype, and its BailingMoeV3 layer calls kda_state_dtype without an SSM override (model_executor/models/bailing_moe_v3.py:611), so --mamba-ssm-cache-dtype does not reach this model's state there; SGLang's --mamba-ssm-dtype bfloat16 halves it.] The split by head is exact while the tensor-parallel size divides the 32 heads: 1, 4 and 8 do and 72 does not.
The split is read off the model's own code, not inferred from the name. modeling_bailing_moe_v3.py lines 1004-1014 select BailingMoeV3MultiLatentAttention when (layer_idx + 1) % layer_group_size == 0 and BailingMoeV3KimiDeltaAttention otherwise; with num_hidden_layers 42 and layer_group_size 6 that is layers 5, 11, 17, 23, 29, 35 and 41, seven in all, and the second clause of the condition (layer_idx >= 42 // 6 × 6 = 42) never fires. The card describes the same thing in words: “5:1 alternating stacking of Kimi Delta Attention (KDA) and MLA”.
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 25 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
Parameter count from the safetensors API rather than a headline, so it is the sum of the dtype split: F8_E4M3 1,009,494,523,904, BF16 15,919,881,728 and F32 19,712, read at commit 2314abdf, whose weight files are unchanged since they were uploaded on 2026-05-14. The API leaves out the 243,446,400 BF16 weight_scale tensors, which the bytes include: 1,041,821,259,008 by the index's metadata.total_size. The multi-token-prediction layer is layer 80, 13,183,264,768 BF16 and 256 F32 parameters in the headers of shards 168 to 170, and config.json declares num_nextn_predict_layers 1. CONFLICT preserved on the active parameter count, which the card, config.json and the report do not state. Ant Group's developer documentation for the model, whose footer names Ant Intelligence (Hang Zhou) Technology Co., Ltd., the provider the card's EU training-content summary names, says “Ring-2.6-1T is a trillion-parameter thinking model with about 63B activated parameters per inference” (last updated 2026-09-23). vLLM's recipe for the same repository says “1T total / 50B active” (added 2026-05-14). 63 is carried because it is the provider's own figure and two more of its statements agree with it: the report says the Ling-2.5/2.6 and Ring-2.5/2.6 families share one base, and the Ring-2.5-1T card says the trillion-scale version of that architecture raised the activated count “from 51B to 63B”. Ring-2.5-1T's config.json matches this one in every architecture field but num_nextn_predict_layers, and its safetensors API gives the same 1,009,494,523,904 F8_E4M3 parameters and 13,183,265,024 fewer BF16 and F32 ones, exactly this checkpoint's multi-token-prediction layer. How the recipe arrived at 50B is not recorded. It is the count InclusionAI's Ring-1T card gives the Ling 2.0 architecture that model adopts, “1 trillion total parameters with 50 billion activated parameters”, and the recipe lists Ring-1T-FP8 among its related recipes. The count is not derived here from expert dimensions. The format is F8_E4M3 through compressed-tensors' FP8_DYNAMIC scheme, one static scale per output channel and activations scaled per token at run time, over a config whose torch_dtype is bfloat16, so the release is graded publisher-quantized: the report says “Ling-2.6 uses FP8 training from pretraining through supervised fine-tuning”, with FP32 master weights, which is how it was trained and not this artefact's per-channel format. Context is config.json's max_position_embeddings; the card lists “128K -> 256K (YaRN)”. Released 2026-05-14, the day the repository and vLLM's recipe appeared; the EU summary dates the model's placement on the Union market 2026-05-11.
kv_lora_rank 512 plus qk_rope_head_dim 64 gives 576 elements per token per layer on the ten latent layers: 11.52 MB per 1,000 tokens at bf16, where Kimi K3 keeps 27.65 at 2.7 times the parameters. The 70 Lightning Attention layers add no slope. Each keeps a state of num_attention_heads 64 × head_dim 128 × head_dim 128 = 1,048,576 elements per sequence, the shape both runtimes allocate, and no convolution window, because the layer has no short convolution (modeling_bailing_moe_v2_5.py 705-759). SGLang fa663e72, which this layout names, builds it as a Mamba-2 state with no channels and conv_kernel 1 (srt/configs/bailing_hybrid.py 217-225), so the window holds no inputs. It passes no dtype with that state (227), so the state takes the default, which reads the environment variable SGLANG_MAMBA_SSM_DTYPE alone and is float32 without it (srt/configs/mamba_utils.py 95, 112); the checkpoint's config.json names no mamba_ssm_dtype in any case. That is 4,194,304 bytes per layer, 293.60 MB per sequence across the 70. Ant Group's own ling_2_5 branch, which the card installs, builds the same shape at the same default. vLLM e52be1a6 allocates the same shape at the model dtype, bf16 (model_executor/layers/mamba/mamba_utils.py 55-61), so half as many bytes. That constant is what the layer split costs: 293.60 MB is what the ten latent layers cache for about 25,500 tokens at bf16. The split by head is exact while the tensor-parallel size divides the 64 heads: 1, 2, 4 and 8 do and 72 does not. The multi-token-prediction layer keeps a latent cache of its own, which belongs to a speculative draft and is not counted.
The split is read off the model's own code, not inferred from the name. modeling_bailing_moe_v2_5.py lines 979-989 select BailingMoeV2_5MultiLatentAttention when (layer_idx + 1) % layer_group_size == 0 and BailingMoeV2_5LinearAttention otherwise; with num_hidden_layers 80 and layer_group_size 8 that is layers 7, 15, 23, 31, 39, 47, 55, 63, 71 and 79, ten in all, and the second clause of the condition (layer_idx >= 80 // 8 × 8 = 80) never fires inside the 80 decoder layers. The linear layers are Lightning Attention-2 through flash-linear-attention's simple-GLA kernels (705-759). The report says the same in words: Lightning Attention and MLA “in a 7:1 hybrid ratio”.
config.json: num_experts 256, num_experts_per_tok 8, num_shared_experts 1, and first_k_dense_replace 4, so the first four of the 80 layers run a dense feed-forward block instead. The report's section 2.1.1 says the same: 1 shared and 256 routed experts, 8 activated per token, and dense layers for the first 4 blocks.
Ring-2.6-1T (xhigh), the only configuration Artificial Analysis lists for it, with inclusionAI/Ring-2.6-1T, the repository this entry sizes, named as its weights: the page prints “scores 17” (16.62 unrounded), a measurement.
Parameter count from the safetensors API (BF16 201,365,304,064 plus F32 12,096) rather than the card's rounded 198B. Active parameters are the card's “approximately 11B parameters per token”, stated as an approximation there and carried as one here; config.json gives moe_num_experts 288 and moe_top_k 8 but no active total.
2 (key and value) × num_attention_groups 8 × head_dim 128 = 2,048 elements per token per layer, the same on both kinds. The twelve global layers are the whole slope: 49.15 MB per 1,000 tokens at bf16. The thirty-three sliding layers stop at 512 tokens and then hold a fixed 69.21 MB per sequence. Past a few thousand tokens the window is doing almost all of the work — an uncapped reading of this same geometry would be 180.00 KiB per token against the 48.00 KiB per token modelled here. Eight key-value groups means the cache stops dividing beyond eight accelerators and replicates thereafter.
text_config.layer_types lists 48 entries, 12 of them full_attention at indices 0, 4, 8 ... 44 and the rest sliding_attention. num_hidden_layers is 45 and num_nextn_predict_layers is 3, so indices 45 to 47 are speculative-decoding layers and the model that answers has 12 full and 33 sliding, not 36. Both attention types use num_attention_groups 8 and head_dim 128; attention_other_setting raises the query heads from 64 to 96 on the sliding layers and leaves the key-value geometry alone. sliding_window is 512.
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 19 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
119.40B is the safetensors API's dtype split summed (F8_E4M3 117,879,865,344 plus BF16 1,521,452,032), which agrees with the card's “119B parameters, with 6.5B activated per token”. The weights figure follows the same split at one byte and two bytes respectively rather than assuming a single precision.
kv_lora_rank 256 plus qk_rope_head_dim 64 = 320 cached elements per token per layer across all 36 layers: 23.04 MB per 1,000 tokens at bf16, or 22.50 KiB per token. Mistral Large 3 caches 576 elements on 61 layers, so the smaller model here is cheaper per token of context by a factor of three from geometry alone. As with every latent model the cache is a single head, so ordinary tensor parallelism replicates it on every rank while the weights divide.
text_config: kv_lora_rank 256, qk_rope_head_dim 64, q_lora_rank 1024, num_hidden_layers 36, sliding_window null and no layer_types key, so every layer is the same and every layer grows.
TWO PARAMETER COUNTS, AND THIS FIELD HOLDS THE CHECKPOINT ONE. DeepSeek states 552B backbone parameters and 196B Engram parameters as a pair and never sums them; the repository dtype tally for the same files gives 763,205,315,794, which is backbone plus Engram plus the vision tower plus the three DSpark draft layers plus embeddings. The 15.2B that 552 + 196 leaves unaccounted for is not itemised by the publisher and is not guessed here. The checkpoint figure is stored for the reason Qwen3.8-Flash-Next stores 180B rather than its 125B of transformer: a server loads what is in the file. It is also the only one the release is self-consistent with — 510.29 GB over 763.21B is 0.669 bytes per parameter, where a mixed FP4/FP8 release should land, while 552B would put it at 0.924, an outlier against every 4-bit checkpoint in this catalog. ACTIVE PARAMETERS ARE PHASE-DEPENDENT AND THIS FIELD IS NOT. The report states 8B active per prefill token and 16B per decode token, a consequence of the causal encoder-decoder projecting the decoder KV from the encoder final hidden state. 16 is stored because the bandwidth floor this figure feeds is a decode model; the 8B prefill figure is the more distinctive of the two and is recorded here rather than dropped. Artificial Analysis independently lists 552 and 16 for the same model. WHERE SERVING RECIPES KEEP THE ENGRAM TABLES, read 2026-10-07 and not modelled by the memory fit, which still places all 510.29 GB of the checkpoint on accelerators and so over-states what a server needs there wherever a recipe moves the tables. Both engines can keep them in pinned host memory instead of HBM, and neither does by default: vLLM e52be1a6 defaults engram-config cpu_offload to false (vllm/config/engram.py:38-44) and SGLang 0b635266 defaults SGLANG_ENABLE_DSV41_ENGRAM_HOST_TABLE to false (srt/environ.py:1576-1578). InferenceX at 18b94249 turns them on for every NVIDIA system whose recipes for this model it holds: its vLLM recipes for H100, H200, B200 and B300, and those for GB200 and GB300, set cpu_offload to true in every variant and say in their headers that Engram weights use UVA DRAM while the KV cache is GPU-resident (benchmarks/single_node/srt-slurm-recipes/dsv41flash/vllm/h200-fp4-mtp/agentic.yaml:2-3, 39), and its SGLang recipes for the same six set the host table with one shard per rank, except two GB300 variants, pure tensor-parallel 4 at concurrency 1 and 2 (dsv41flash/sglang/gb300-fp4-mtp/agentic.yaml:65-68). For AMD it keeps the tables on the GPU at the widest tensor-parallel setting each card runs and moves them to host memory below it (configs/amd-master.yaml:1064-1066). The 511 GB checkpoint, less the offloaded tables, fits in 8x141 GB on H200, the one arm that says what that buys (configs/nvidia-master.yaml:8742-8743), and the SGLang H200 recipe puts the HBM freed at about 23 GiB per GPU (sglang/h200-fp4-mtp/agentic.yaml:67-68). These are one third party's benchmark configurations: they show that servers run this model with its tables in host memory, not what that costs in speed, and whether the fit should model weights a server can keep in host memory is not decided.
Read from inference/model.py in the release repository, and its geometry reproduces the technical report headline figure exactly at the reference implementation's own widths, which is the check that makes the element counts trustworthy rather than plausible. THE LAYER COUNTS HERE ARE BUFFERS, NOT LAYERS THAT ATTEND. config.json lists compress_ratios over 43 layers as 5 at 0, 18 at 2 and 20 at 1, but Attention registers compress_kv_cache only when layer_id is in kv_source_layer_ids [2, 8, 14, 20], and every other layer reads those; the file says so in a comment. So four buffers exist: three at head_dim 512 over ratio 2 = 256 elements per token, and one at 512 over 1 = 512, a 1,280-element slope. THE INDEXER OWNS KEYS ON THE SAME FOUR LAYERS, NOT ON ALL EIGHT index_source_layer_ids: Indexer sets owns_k from kv_source_layers and registers k_cache only then, because the index keys are derived from the compressor latent and only a layer that compresses its own KV can produce them. That is 3 x 128/2 + 1 x 128/1 = 320 elements per token. THE REFERENCE IMPLEMENTATION'S WIDTHS ARE PER BUFFER: its latent goes through fp4_act_quant with a block of 16 and an e4m3 scale, so 0.5 bytes plus one scale byte per 16 elements = 0.5625; its index keys use the module-wide fp4_block_size of 32, so 0.53125. 1,280 x 0.5625 + 320 x 0.53125 = 720 + 170 = 890 bytes per token, which is the figure the report states. [CORRECTED 2026-10-07: the two compressed groups carried bytesPerElementOverride 0.5625 and the two index groups 0.53125, whatever Cache precision said, so the entry held 890 bytes per token at fp8 and at bf16 alike. That is the reference implementation's storage, and a deployment runs an engine: the engines this entry cites store a compressed entry in 584 bytes and an index key in 132 by default, so the entry now records 1,790 bytes per token at fp8 and 2,890 at bf16, as the paragraphs below show.] THE WINDOW is registered on all 43 layers at [batch, 128, head_dim] as window_kv_cache, and the report says FP8 is retained for it. 43 x 128 x 512 = 2,818,048 elements, which is 5.64 MB per sequence on the bf16 basis every note in this file quotes for comparability. AT FP8 AN ENTRY OF THE WINDOW IS 584 BYTES, NOT 512. vLLM e52be1a6 resolves the window's dtype with the same function as the compressed cache's (models/deepseek_v4_1/attention.py:112-147, 445-456), and the default FlashMLA attention class sets use_fp8_ds_mla_layout (attention.py:170; nvidia/model.py:114-149), so even at --kv-cache-dtype auto the window is DeepSeek's fp8_ds_mla layout: 448 fp8 values, the 64 rotary elements at bf16 and 8 bytes of scales and padding, 584 bytes (v1/attention/backends/mla/sparse_swa.py:107-124); that path refuses a bf16 cache (attention.py:129-135). Only the FlashInfer sparse backend for SM100, which has to be named, keeps a plain row, 512 bytes at fp8 and 1,024 at bf16 (nvidia/flashinfer_sparse.py:198). A later vLLM head, 08567505, writes DeepSeek's V4.1 record, 528 bytes (512 fp8 values and one ue8m0 scale per 32), on SM100 and keeps the 584-byte record on every other architecture (models/deepseek_v41/attention.py:122-129, 513). SGLang fa663e72 has no kv_layout module and its pool asserts the 584-byte record (srt/mem_cache/deepseek_v4_memory_pool.py:117-135), so its V4.1 layouts are read at 0b635266: SGLANG_DSV4_KV_LAYOUT defaults to v4, 584 bytes on every GPU, and the 528-byte layout is built only when v41 is asked for or auto finds SM100 (srt/environ.py:1562-1565; deepseek_v4_memory_pool.py:165-198; kernels/ops/attention/dsv4/kv_layout.py:16-24, 54-56), the window's pool being built from the main layout and the same pool class as the compressed pools (deepseek_v4_memory_pool.py:1080, 1103, 1165; srt/model_executor/pool_configurator.py:1261-1296). Neither engine stores the window at four bits: vLLM's nvfp4_ds_mla record keeps the 528-byte window and shrinks the compressed cache alone (08567505 attention.py:514-518), and SGLang's fp4 layout exists only as the extra cache beside a V4.1 main cache (kv_layout.py:93-96). The one wider layout, the unified Triton pool, keeps bf16 rows whatever the cache dtype but is built on HIP only, behind SGLANG_HACK_FLASHMLA_BACKEND (deepseek_v4_memory_pool.py:1123-1137; kernels/ops/attention/dsv4/unified_kv_kernels/env_gate.py:12-15; environ.py:1569-1574), and its pool refuses a model with kv-source layers at ratio 1 or 2, as this one has (assert not self._unified_kv, deepseek_v4_memory_pool.py:1450-1454), so unlike DeepSeek V4 there is nothing to leave out. Recording the larger of what the engines store gives 1,024 bytes at bf16, as before, and 584 at fp8, so the window carries fp8BytesPerElement 1.140625 and holds 3,214,336 bytes per sequence at the fp8 this profile defaults to. [CORRECTED 2026-10-07: this left the window on the model default of one byte at fp8, 2,818,048 bytes per sequence, 12.3% below the 584-byte layout both engines store by default.] THE COMPRESSED LATENT FOLLOWS CACHE PRECISION AT THE SAME 584 BYTES, AND THE INDEX KEYS DO NOT. vLLM e52be1a6 sizes a compressed entry of a kv-source layer in the same fp8_ds_mla record, 584 bytes per compress_ratio tokens, at the dtype the same resolver returns (models/deepseek_v4_1/attention.py:930-954), so the default FlashMLA class stores it at 584 even at auto and refuses bf16 (129-135), and the FlashInfer sparse class keeps a plain row, 512 bytes at fp8 and 1,024 at bf16 (143-147). SGLang 0b635266 builds both compressed pools from DeepSeekV4SingleKVPool at the layout resolve_compressed_kv_layout returns, which under its default v4 main layout is the 584-byte V4 layout (srt/mem_cache/deepseek_v4_memory_pool.py:139-162, 1456-1486). Recording the larger gives 1,024 bytes at bf16 and 584 at fp8, so the two compressed groups carry fp8BytesPerElement 1.140625 and no override, as DeepSeek V4's latent groups do. The index keys are fixed whatever the dtype. vLLM e52be1a6 allocates one 132-byte row per compress_ratio tokens, 128 fp8 values and one fp32 scale (attention.py:82-89, 375-384, 991-1003), and the 68-byte MXFP4 row only when the operator sets indexer_kv_dtype to mxfp4 on an SM100 GPU (v1/attention/backends/mla/indexer.py:58-73; config/attention.py:83-87, 135-139); it reads no model field for that, and this model's config.json has none, its indexer fields being index_n_heads, index_head_dim 128 and index_topk. SGLang 0b635266 forces the 68-byte fp4 layout on the ratio 1 and ratio 2 indexer pools, the only ones this model has (deepseek_v4_memory_pool.py:43-47, 1502-1514, 1577-1591). The larger is vLLM's 132 bytes, 1.03125 per element as for DeepSeek V4's keys, so the index groups keep bytesPerElementOverride with that value. That gives 1,280 x 1.140625 + 320 x 1.03125 = 1,460 + 330 = 1,790 bytes per token at fp8, the profile's default, and 1,280 x 2 + 320 x 1.03125 = 2,560 + 330 = 2,890 at bf16. NARROWER LAYOUTS EXIST AND ARE NOT MODELLED. vLLM 08567505 makes the 288-byte NVFP4 compressed record, which is the reference implementation's format, and the 528-byte window record the default on SM100 wherever its mega attention class can serve the topology, because that class resolves an unspecific --kv-cache-dtype to nvfp4_ds_mla (models/deepseek_v41/nvidia/model.py:234-241; nvidia/flash_mla_mega_attn.py:201-205; attention.py:516-518). SGLang builds the 288-byte latents and the 528-byte window when SGLANG_DSV4_KV_LAYOUT is v41, or auto on SM100 (srt/environ.py:1562-1568; deepseek_v4_memory_pool.py:165-198). On SM100 with either, the compressed entries take about half the bytes recorded here, and the entry records the larger by the rule that makes a fit hold under either engine. RoPE occupies the last 64 of the 512 rather than adding to them, the same slice convention as V4, so an entry is 512 elements and not 576. READING LAYERS ARE NOT BUFFERS EITHER: every layer with a compress ratio above zero reads the buffer of the nearest kv source at or below it (inference/model.py:739-763; vLLM e52be1a6 models/deepseek_v4_1/attention.py:284-289), so 18 layers read the three 2:1 buffers and 20 read the 1:1 one, and each of the eight index_source_layer_ids scores every key its kv source stores (model.py:554-557), so the 2:1 keys are read by 3 layers and layer 20's by 5. Layers 0 and 1 and the three DSpark layers read only their windows. The groups record those counts as readingLayers, which multiply what a step reads and not what memory holds. [CORRECTED 2026-09-13: both bandwidth searches charged each buffer once, so the estimate read 890 bytes per token of each copy where a step reads 8,794, at the widths recorded then.] At the widths recorded now a step reads 17,794 bytes per token at fp8: 18 x 292 + 20 x 584 + 3 x 66 + 5 x 132. [STILL EXCLUDED: whether the hierarchical indexer candidate pool (candidate_topk_blocks 2048 at candidate_block_size 8) needs storage beyond the index keys already counted. The release does not say, and the difference between the readings would be larger than the term.]
The release inference/model.py states the trap in a comment of its own: a compress_ratio above zero does not mean the layer compresses its own KV, only kv_source_layers do and the rest read that same cache. A causal encoder-decoder splits the 40 backbone layers into a 20-layer encoder and a 20-layer decoder, which is why 8B of parameters are active per prefill token and 16B per decode token. WHAT A STEP READS: index_topk counts compressed entries, and the Indexer keeps topk(min(index_topk, end_pos // ratio)) of them (inference/model.py:578), so a ratio-2 layer reads at most 512 entries of two tokens each, which is the 2:1 group's tokensPerEntry, and a ratio-1 layer 512 entries of one token. [CORRECTED 2026-09-13: selectedEntries was selectedTokens, and the bandwidth lower bound charged the 2:1 buffers 512 tokens rather than 512 entries.]
config.json text_config: n_routed_experts 384, num_experts_per_tok 6, n_shared_experts 1 — the same expert shape as V4 Pro. The three DSpark draft layers run a lighter stack of their own (dspark_n_routed_experts 128, dspark_num_experts_per_tok 3) which is not modelled here.
Released 2026-07-24 at the same price as Opus 4.8. Cache write field uses the five-minute tier, and the one-hour field the hour-long one. Anthropic documents the full 1M window at flat pricing, so no long-context surcharge applies. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128k for Claude Opus 5. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $10 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share. [RETIRED 2026-09-25, superseded by Claude Opus 5.5. Anthropic still sells Claude Opus 5 at these rates, and its deprecations page still lists claude-opus-5 as “Active”, with retirement “Not sooner than July 24, 2027”. But its models overview, read the same day, lists Claude Opus 5 under “Legacy models (still available)” and tells a reader to “start with Claude Opus 5.5 for most workloads”, and Anthropic's announcement calls Opus 5.5 “a major step up from Opus 5”. Opus 5.5, released 2026-09-22, also costs less on every rate this row carries: $4 / $20 / $0.20 / $5 for input, output, cache read and five-minute cache write against $5 / $25 / $0.50 / $6.25, with the same window and output caps. The lifecycle definition asks what a buyer would choose today; still being on sale does not answer that, and a buyer choosing between the two takes the cheaper, newer one.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Official Batch API 50% multiplier; cache write uses the five-minute tier. New-generation tokenizer (~30% more tokens than Sonnet 4.6). OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and 300,000 with the output-300k-2026-03-24 beta header. Anthropic's thinking page prints 128k for Claude Opus 5 and a batches beta ceiling of 300k, and its batch-processing page names Claude Opus 5 among the models for whose batch requests that header “raises the `max_tokens` cap to 300,000”, on the Message Batches API only. An output between the two caps is in doubt rather than refused, because a scenario does not record whether a request sends the header. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward both caps. ONE-HOUR WRITE, recorded 2026-09-25: $10 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. [RETIRED 2026-09-25, superseded by Claude Opus 5.5. Anthropic still sells Claude Opus 5 at these rates, and its deprecations page still lists claude-opus-5 as “Active”, with retirement “Not sooner than July 24, 2027”. But its models overview, read the same day, lists Claude Opus 5 under “Legacy models (still available)” and tells a reader to “start with Claude Opus 5.5 for most workloads”, and Anthropic's announcement calls Opus 5.5 “a major step up from Opus 5”. Opus 5.5, released 2026-09-22, also costs less on every rate this row carries: $4 / $20 / $0.20 / $5 for input, output, cache read and five-minute cache write against $5 / $25 / $0.50 / $6.25, with the same window and output caps. The lifecycle definition asks what a buyer would choose today; still being on sale does not answer that, and a buyer choosing between the two takes the cheaper, newer one.] [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Still offered and supported — Anthropic lists it under “Legacy models (still available)” on its models overview, which is where that heading lives and is now cited rather than the pricing page this row used to point at for it with no deprecation notice — but superseded as the recommended choice by Opus 5 on 2026-07-24 at an identical price, so it is hidden by default. Cache write field uses the five-minute tier. New-generation tokenizer (~30% more tokens than Sonnet 4.6). OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128k for Claude Opus 4.8. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $10 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share. REPOINTED 2026-09-25: its replacement moves from Claude Opus 5 to Claude Opus 5.5, because Claude Opus 5 is now retired itself and a replacement has to be a live tier, or the calculator's switch to the current equivalent would land on another retired one.
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Cache write field uses the five-minute tier, and the one-hour field the hour-long one; batch is in claude-fable-5-batch. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128k for Claude Fable 5. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share. [RETIRED 2026-09-25, superseded by Claude Fable 5.1. Anthropic still sells Claude Fable 5 at these rates, and its deprecations page still lists claude-fable-5 as “Active”, with retirement “Not sooner than June 9, 2027”. But its models overview, read the same day, lists Claude Fable 5 under “Legacy models (still available)”, and its migration guide says “Claude Fable 5.1 succeeds Claude Fable 5 at the same input and output prices, with cache reads at a quarter of the cost.” Anthropic's own benchmark table puts Fable 5.1 ahead of Fable 5 on every row it prints. The lifecycle definition asks what a buyer would choose today, and at equal or lower rates that is Fable 5.1. This catalog kept Fable 5 current when Fable 5.1 was added, on the ground that Anthropic's pricing table listed both with no deprecation tag, which tested whether it was still sold rather than whether anyone would choose it.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Still offered under “Legacy models (still available)” on Anthropic's models overview with no deprecation notice, but superseded by Sonnet 5 (2026-06-30), so it is hidden by default. Cache write field uses the five-minute tier. Last generation on the older tokenizer. [CORRECTED 2026-08-29: this note used to argue that the older tokenizer made its “nominally identical $3/$15 cheaper in practice than the Sonnet 5 standard rate”. Both halves were wrong and together they reversed the conclusion. Sonnet 5's standard rate is $2/$10, in this catalog's own claude-sonnet-5 row and on Anthropic's page; the only $3/$15 is claude-sonnet-5-standard, which this catalog itself describes as a price that never took effect. And the tokenizer difference is about 23%, not 30%: the newer tokenizer produces about 30% more tokens for the same text, so the older one produces 1/1.30 of them. $3 over 0.77 as many tokens is an effective $2.31 against Sonnet 5's $2.00, so Sonnet 4.6 is about 15% dearer for the same text, not cheaper. The heading quoted above was also cited to the pricing page, which does not carry it.] OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128k for Claude Sonnet 4.6. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $6 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share.
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort), the configuration every other Claude tier here is scored at: the page prints “scores 30”, a measurement. [CORRECTED 2026-09-26: this read 25, from the page at /models/claude-sonnet-4-6, which is Claude Sonnet 4.6 (Non-reasoning, High Effort) and prints “scores 25 (estimated)”. Its address carries no suffix, like the max-effort pages of Claude Sonnet 5 and Claude Opus 5.5, but it describes Sonnet 4.6 with reasoning off.]
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Official Batch API 50% multiplier; cache write uses the five-minute tier. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. Retired 2026-08-15: Claude Opus 5 Batch charges exactly the same /5 and scores 63 against 57 on the same index, so there is no workload where this tier is the better buy. The standard 4.8 tier was retired at the Opus 5 launch and this one was left behind — a model is not half-retired. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and 300,000 with the output-300k-2026-03-24 beta header. Anthropic's thinking page prints 128k for Claude Opus 4.8 and a batches beta ceiling of 300k, and its batch-processing page names Claude Opus 4.8 among the models for whose batch requests that header “raises the `max_tokens` cap to 300,000”, on the Message Batches API only. An output between the two caps is in doubt rather than refused, because a scenario does not record whether a request sends the header. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward both caps. ONE-HOUR WRITE, recorded 2026-09-25: $10 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. REPOINTED 2026-09-25: its replacement moves from Claude Opus 5 to Claude Opus 5.5, because Claude Opus 5 is now retired itself and a replacement has to be a live tier, or the calculator's switch to the current equivalent would land on another retired one. [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Official Batch API 50% multiplier; cache write uses the five-minute tier. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and no higher cap on the Batch API. Anthropic's thinking page prints 128k for Claude Fable 5 and a dash in its batches beta ceiling column, and its batch-processing page does not name Claude Fable 5 among the models the output-300k-2026-03-24 header raises. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. [RETIRED 2026-09-25, superseded by Claude Fable 5.1. Anthropic still sells Claude Fable 5 at these rates, and its deprecations page still lists claude-fable-5 as “Active”, with retirement “Not sooner than June 9, 2027”. But its models overview, read the same day, lists Claude Fable 5 under “Legacy models (still available)”, and its migration guide says “Claude Fable 5.1 succeeds Claude Fable 5 at the same input and output prices, with cache reads at a quarter of the cost.” Anthropic's own benchmark table puts Fable 5.1 ahead of Fable 5 on every row it prints. The lifecycle definition asks what a buyer would choose today, and at equal or lower rates that is Fable 5.1. This catalog kept Fable 5 current when Fable 5.1 was added, on the ground that Anthropic's pricing table listed both with no deprecation tag, which tested whether it was still sold rather than whether anyone would choose it.] [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Official Batch API 50% multiplier; cache write uses the five-minute tier. Superseded by the Sonnet 5 batch tier. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and 300,000 with the output-300k-2026-03-24 beta header. Anthropic's thinking page prints 128k for Claude Sonnet 4.6 and a batches beta ceiling of 300k, and its batch-processing page names Claude Sonnet 4.6 among the models for whose batch requests that header “raises the `max_tokens` cap to 300,000”, on the Message Batches API only. An output between the two caps is in doubt rather than refused, because a scenario does not record whether a request sends the header. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward both caps. ONE-HOUR WRITE, recorded 2026-09-25: $6 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort), the configuration every other Claude tier here is scored at: the page prints “scores 30”, a measurement. [CORRECTED 2026-09-26: this read 25, from the page at /models/claude-sonnet-4-6, which is Claude Sonnet 4.6 (Non-reasoning, High Effort) and prints “scores 25 (estimated)”. Its address carries no suffix, like the max-effort pages of Claude Sonnet 5 and Claude Opus 5.5, but it describes Sonnet 4.6 with reasoning off.]
CHANGED at the 2026-08-13 refresh: this is now the permanent standard rate. Anthropic: "The $2/$10 per million input/output token pricing for Claude Sonnet 5, announced at launch as introductory pricing through August 31, 2026, is now the standard price. The previously scheduled increase to $3/$15 per million input/output tokens on September 1, 2026 will not occur." The expiry this catalog previously carried has therefore been removed. Five-minute cache-write tier (one-hour is $4/M). New-generation tokenizer (~30% more tokens than Sonnet 4.6). OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128k for Claude Sonnet 5. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $4 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share.
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
A price that never took effect. Anthropic scheduled Sonnet 5 to rise to $3/$15 on 2026-09-01, this catalog encoded it so multi-year scenarios could be priced honestly against it, and Anthropic then cancelled the increase. Kept only so scenario URLs shared while the increase was still scheduled continue to resolve; it is hidden by default and should not be used for new planning. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128k for Claude Sonnet 5. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap.
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Now the permanent standard rate with the official Batch API 50% multiplier — Anthropic cancelled the 2026-09-01 increase that this tier's expiry previously anticipated. Five-minute cache-write tier. New-generation tokenizer (~30% more tokens than Sonnet 4.6). OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and 300,000 with the output-300k-2026-03-24 beta header. Anthropic's thinking page prints 128k for Claude Sonnet 5 and a batches beta ceiling of 300k, and its batch-processing page names Claude Sonnet 5 among the models for whose batch requests that header “raises the `max_tokens` cap to 300,000”, on the Message Batches API only. An output between the two caps is in doubt rather than refused, because a scenario does not record whether a request sends the header. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward both caps. ONE-HOUR WRITE, recorded 2026-09-25: $4 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Graded estimated rather than official-claim: two of the four figures here are read off Anthropic's batch table and two are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. The batch half of a price increase Anthropic scheduled for 2026-09-01 and then cancelled. Retained only so older shared scenarios resolve. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and 300,000 with the output-300k-2026-03-24 beta header. Anthropic's thinking page prints 128k for Claude Sonnet 5 and a batches beta ceiling of 300k, and its batch-processing page names Claude Sonnet 5 among the models for whose batch requests that header “raises the `max_tokens` cap to 300,000”, on the Message Batches API only. An output between the two caps is in doubt rather than refused, because a scenario does not record whether a request sends the header. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward both caps. [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
OUTPUT CAP, recorded 2026-09-13: 64,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 64k for Claude Haiku 4.5, with its batches beta ceiling “Not available”. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. RE-READ 2026-09-25: prices and caps unchanged. Anthropic's models overview gives this model's retirement as “Not sooner than October 15, 2026”, the nearest date of any current Claude model, so a multi-year horizon costed against this tier assumes a rate Anthropic has committed to for another 20 days. That is a floor, not an announcement: no deprecation has been published for it. ONE-HOUR WRITE, recorded 2026-09-25: $2 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share.
Claude Haiku 4.5 predates the parameter. "Requests with inference_geo on Claude Opus 4.5, Claude Sonnet 4.5, Claude Haiku 4.5, or earlier models return a 400 error" — so this tier cannot be pinned to a jurisdiction at any price, which is a different answer from the surcharge being unknown. Recorded so the calculator can say which. Read 2026-09-10.
Claude 4.5 Haiku (Reasoning), the one configuration with reasoning on of the two Artificial Analysis lists: the page prints “scores 17”, a measurement. [CORRECTED 2026-09-26: this read 15, from the page at /models/claude-4-5-haiku, which is Claude 4.5 Haiku (Non-reasoning) and prints “scores 15 (estimated)”, while every other Claude tier here is scored with reasoning on.]
CUT 2026-08-21 to $4.00 / $0.40 / $5.00 / $20.00 from $5.00 / $0.50 / $6.25 / $30.00 — 20% off input and 33.3% off output. OpenAI's API changelog, entry dated Aug 21 and tagged gpt-5.6-sol: "GPT-5.6 Sol now costs $4 per million input tokens and $20 per million output tokens, representing 20% lower input pricing and 33% lower output pricing." The cache write stays at 1.25x input, which is what the reversal below established. Deliberately carries no effectiveUntil. OpenAI's own wording is "GPT-5.6 Sol's promotional pricing is available at least through November 21, 2026" — a floor on how long the rate lasts, not a date it stops. Writing 2026-11-21 into effectiveUntil would assert an increase the vendor has not announced. [CORRECTED 2026-08-27: this note also claimed the picker would then hide a live tier, which is false about this repository's own code — the picker filters on whether a successor row exists (listPriceTierFor), never on a date, so a tier with an end date and no successor stays visible. The real cost of leaving the field empty is the opposite and worth stating: no promotional-pricing caution fires, so a multi-year break-even against this tier is computed against a rate the vendor guarantees only to 2026-11-21 and says nothing about afterwards. Contrast GLM-5.3-Flash, where Z.ai wrote "the promotion ends at 24:00 on September 9, 2026" and effectiveUntil is the right field.] Terra and Luna were re-read from the same table on 2026-08-27 and are unchanged at $2.00/$12.00 and $0.20/$1.20. Explicitly excluded from the 2026-07-30 price cut that reduced Terra 20% and Luna 80%. The >272K long-context multipliers apply to input, cached input, cache write and output alike. [CORRECTED 2026-08-15: this tier carried a cache-write rate of $6.25/M, 1.25x its input rate, copied from Anthropic’s billing model. OpenAI does not bill a cache write at all — caching is automatic, the first call pays the ordinary input rate and matched prefixes pay the cached rate afterwards. The write bucket is therefore priced at the plain input rate, which is what actually happens. Corroborated by the owner’s own Codex telemetry, where the cache-write counter is structurally zero across 357 sessions and 6.12 billion input tokens.] [REVERSED 2026-08-21, and this is the important half of the note. The "correction" above was wrong: OpenAI does bill a cache write on this model family, at exactly the 1.25x multiplier it was accused of borrowing from Anthropic. Their caching guide states it outright. Under an expander headed “GPT-5.6 and later”: “For GPT-5.6 and later, cache writes cost 1.25× the standard, uncached input-token rate.” Its comparison table says the same in a Cache write charge row, reading “1.25× the uncached input-token rate” for that family against “No additional cache-write charge” for the two older ones.] [CORRECTED 2026-08-27: the sentence quoted here until today was a paraphrase in quotation marks — none of “have no additional fee”, “billed at 1.25” or “reported in cache_write_tokens” appears anywhere on that page. It said the right thing and it was not a quotation, which on a site whose premise is a citable audit trail is its own defect. The same correction removes the absolute figures that followed it — $6.25, $2.50 and $0.25 against inputs of $5.00, $2.00 and $0.20 — because that was the table as it read on 2026-08-21 and Sol was cut on the same day. The ratio is what survived. The telemetry that seemed to corroborate the error is not wrong either; it is pre-GPT-5.6 traffic, on models where the fee genuinely did not exist, and it was generalised to a family that had introduced one on 2026-07-09. A measurement of the wrong thing is more persuasive than no measurement at all, which is exactly why it took a source read to catch.] TOKEN LIMITS, recorded 2026-09-13: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-5.6-sol page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.” [RETIRED 2026-09-25, superseded by GPT-6 Sol, by the lifecycle definition rather than by anything OpenAI has announced. OpenAI has neither deprecated GPT-5.6 Sol nor tagged it legacy: its deprecations page still names gpt-5.6-sol as the replacement for models it shuts down on 2026-10-23 and 2026-12-11, and its model catalogue still describes it as a “Flagship model for complex professional work”. But GPT-6 Sol, released 2026-09-22, costs exactly half of this model on all four rates, $2.00 / $0.20 / $2.50 / $10.00 against $4.00 / $0.40 / $5.00 / $20.00, with the same window, input limit, output cap and 272K long-context step, and the model index no longer features GPT-5.6 Sol: it opens “If you're not sure where to start, use GPT-6 Astra” and features GPT-5.6 Terra and GPT-5.6 Luna after it. Artificial Analysis marks GPT-5.6 Sol deprecated and scores it 46.97 against GPT-6 Sol's 47.53 on v4.3.2. The definition asks what a buyer would choose today, and nobody pays twice as much for the same limits from the same vendor.]
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists this model as eligible, and every OpenAI tier in this catalog is priced from 2026-07-30 or later, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-10.
Cut 20% on 2026-07-30 from $2.50/$15.00. OpenAI: "Starting July 30, API pricing is $2 per million input tokens and $12 per million output tokens for Terra." The >272K long-context multipliers apply to input, cached input, cache write and output alike. [CORRECTED 2026-08-15: this tier carried a cache-write rate of $2.5/M, 1.25x its input rate, copied from Anthropic’s billing model. OpenAI does not bill a cache write at all — caching is automatic, the first call pays the ordinary input rate and matched prefixes pay the cached rate afterwards. The write bucket is therefore priced at the plain input rate, which is what actually happens. Corroborated by the owner’s own Codex telemetry, where the cache-write counter is structurally zero across 357 sessions and 6.12 billion input tokens.] [REVERSED 2026-08-21, and this is the important half of the note. The "correction" above was wrong: OpenAI does bill a cache write on this model family, at exactly the 1.25x multiplier it was accused of borrowing from Anthropic. Their caching guide states it outright. Under an expander headed “GPT-5.6 and later”: “For GPT-5.6 and later, cache writes cost 1.25× the standard, uncached input-token rate.” Its comparison table says the same in a Cache write charge row, reading “1.25× the uncached input-token rate” for that family against “No additional cache-write charge” for the two older ones.] [CORRECTED 2026-08-27: the sentence quoted here until today was a paraphrase in quotation marks — none of “have no additional fee”, “billed at 1.25” or “reported in cache_write_tokens” appears anywhere on that page. It said the right thing and it was not a quotation, which on a site whose premise is a citable audit trail is its own defect. The same correction removes the absolute figures that followed it — $6.25, $2.50 and $0.25 against inputs of $5.00, $2.00 and $0.20 — because that was the table as it read on 2026-08-21 and Sol was cut on the same day. The ratio is what survived. The telemetry that seemed to corroborate the error is not wrong either; it is pre-GPT-5.6 traffic, on models where the fee genuinely did not exist, and it was generalised to a family that had introduced one on 2026-07-09. A measurement of the wrong thing is more persuasive than no measurement at all, which is exactly why it took a source read to catch.] TOKEN LIMITS, recorded 2026-09-13: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-5.6-terra page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.”
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists this model as eligible, and every OpenAI tier in this catalog is priced from 2026-07-30 or later, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-10.
Cut 80% on 2026-07-30 from $1.00/$6.00. OpenAI: "GPT-5.6 Luna, our fastest and most affordable model, will cost 80% less." At $0.20/$1.20 this is the cheapest frontier-lab tier in the catalog and the hardest comparator for any self-hosting case to beat. [CORRECTED 2026-08-15: this tier carried a cache-write rate of $0.25/M, 1.25x its input rate, copied from Anthropic’s billing model. OpenAI does not bill a cache write at all — caching is automatic, the first call pays the ordinary input rate and matched prefixes pay the cached rate afterwards. The write bucket is therefore priced at the plain input rate, which is what actually happens. Corroborated by the owner’s own Codex telemetry, where the cache-write counter is structurally zero across 357 sessions and 6.12 billion input tokens.] [REVERSED 2026-08-21, and this is the important half of the note. The "correction" above was wrong: OpenAI does bill a cache write on this model family, at exactly the 1.25x multiplier it was accused of borrowing from Anthropic. Their caching guide states it outright. Under an expander headed “GPT-5.6 and later”: “For GPT-5.6 and later, cache writes cost 1.25× the standard, uncached input-token rate.” Its comparison table says the same in a Cache write charge row, reading “1.25× the uncached input-token rate” for that family against “No additional cache-write charge” for the two older ones.] [CORRECTED 2026-08-27: the sentence quoted here until today was a paraphrase in quotation marks — none of “have no additional fee”, “billed at 1.25” or “reported in cache_write_tokens” appears anywhere on that page. It said the right thing and it was not a quotation, which on a site whose premise is a citable audit trail is its own defect. The same correction removes the absolute figures that followed it — $6.25, $2.50 and $0.25 against inputs of $5.00, $2.00 and $0.20 — because that was the table as it read on 2026-08-21 and Sol was cut on the same day. The ratio is what survived. The telemetry that seemed to corroborate the error is not wrong either; it is pre-GPT-5.6 traffic, on models where the fee genuinely did not exist, and it was generalised to a family that had introduced one on 2026-07-09. A measurement of the wrong thing is more persuasive than no measurement at all, which is exactly why it took a source read to catch.] TOKEN LIMITS, recorded 2026-09-13: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-5.6-luna page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.” [RETIRED 2026-09-25, superseded by GPT-6 Luna, by the lifecycle definition and against OpenAI's current recommendation. OpenAI has neither deprecated GPT-5.6 Luna nor tagged it legacy, and its model index, read the same day, still features it: “Choose GPT-5.6 Terra to balance intelligence and cost, or GPT-5.6 Luna for cost-sensitive, high-volume workloads.” That index features no GPT-6 model but Astra, although GPT-6 Luna was released on 2026-09-22 at half this model's input, cached-input and cache-write rates and $0.50 against $1.20 for output, with the same window, input limit, output cap and long-context step. Artificial Analysis marks GPT-5.6 Luna deprecated and puts the two within a tenth of a point on v4.3.2, 37.32 here against 37.26. For the cost-sensitive, high-volume work OpenAI recommends this model for, a buyer choosing today takes the one at less than half the price, which is the definition's test. What the move gives up is recorded rather than modelled: OpenAI's data-residency table offers gpt-5.6-luna in the United Arab Emirates, where it lists no GPT-6 model, and allows GPT-6 Luna in its European region on Standard processing only.]
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists this model as eligible, and every OpenAI tier in this catalog is priced from 2026-07-30 or later, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-10.
CUT 2026-08-21 to $4.00 / $0.40 / $5.00 / $20.00 from $5.00 / $0.50 / $6.25 / $30.00 — 20% off input and 33.3% off output. OpenAI's API changelog, entry dated Aug 21 and tagged gpt-5.6-sol: "GPT-5.6 Sol now costs $4 per million input tokens and $20 per million output tokens, representing 20% lower input pricing and 33% lower output pricing." The cache write stays at 1.25x input, which is what the reversal below established. Deliberately carries no effectiveUntil. OpenAI's own wording is "GPT-5.6 Sol's promotional pricing is available at least through November 21, 2026" — a floor on how long the rate lasts, not a date it stops. Writing 2026-11-21 into effectiveUntil would assert an increase the vendor has not announced. [CORRECTED 2026-08-27: this note also claimed the picker would then hide a live tier, which is false about this repository's own code — the picker filters on whether a successor row exists (listPriceTierFor), never on a date, so a tier with an end date and no successor stays visible. The real cost of leaving the field empty is the opposite and worth stating: no promotional-pricing caution fires, so a multi-year break-even against this tier is computed against a rate the vendor guarantees only to 2026-11-21 and says nothing about afterwards. Contrast GLM-5.3-Flash, where Z.ai wrote "the promotion ends at 24:00 on September 9, 2026" and effectiveUntil is the right field.] Terra and Luna were re-read from the same table on 2026-08-27 and are unchanged at $2.00/$12.00 and $0.20/$1.20. Sol was explicitly excluded from the 2026-07-30 cut ("Sol pricing remains unchanged"). OpenAI documents Flex tokens as priced at Batch API rates and the Batch discount as a flat 50%, so both tiers share this row. [CORRECTED 2026-08-15: this tier carried a cache-write rate of $6.25/M, 1.25x its input rate, copied from Anthropic’s billing model. OpenAI does not bill a cache write at all — caching is automatic, the first call pays the ordinary input rate and matched prefixes pay the cached rate afterwards. The write bucket is therefore priced at the plain input rate, which is what actually happens. Corroborated by the owner’s own Codex telemetry, where the cache-write counter is structurally zero across 357 sessions and 6.12 billion input tokens.] [REVERSED 2026-08-21, and this is the important half of the note. The "correction" above was wrong: OpenAI does bill a cache write on this model family, at exactly the 1.25x multiplier it was accused of borrowing from Anthropic. Their caching guide states it outright. Under an expander headed “GPT-5.6 and later”: “For GPT-5.6 and later, cache writes cost 1.25× the standard, uncached input-token rate.” Its comparison table says the same in a Cache write charge row, reading “1.25× the uncached input-token rate” for that family against “No additional cache-write charge” for the two older ones.] [CORRECTED 2026-08-27: the sentence quoted here until today was a paraphrase in quotation marks — none of “have no additional fee”, “billed at 1.25” or “reported in cache_write_tokens” appears anywhere on that page. It said the right thing and it was not a quotation, which on a site whose premise is a citable audit trail is its own defect. The same correction removes the absolute figures that followed it — $6.25, $2.50 and $0.25 against inputs of $5.00, $2.00 and $0.20 — because that was the table as it read on 2026-08-21 and Sol was cut on the same day. The ratio is what survived. The telemetry that seemed to corroborate the error is not wrong either; it is pre-GPT-5.6 traffic, on models where the fee genuinely did not exist, and it was generalised to a family that had introduced one on 2026-07-09. A measurement of the wrong thing is more persuasive than no measurement at all, which is exactly why it took a source read to catch.] TOKEN LIMITS, recorded 2026-09-13: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-5.6-sol page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.” [RETIRED 2026-09-25, superseded by GPT-6 Sol, by the lifecycle definition rather than by anything OpenAI has announced. OpenAI has neither deprecated GPT-5.6 Sol nor tagged it legacy: its deprecations page still names gpt-5.6-sol as the replacement for models it shuts down on 2026-10-23 and 2026-12-11, and its model catalogue still describes it as a “Flagship model for complex professional work”. But GPT-6 Sol, released 2026-09-22, costs exactly half of this model on all four rates, $2.00 / $0.20 / $2.50 / $10.00 against $4.00 / $0.40 / $5.00 / $20.00, with the same window, input limit, output cap and 272K long-context step, and the model index no longer features GPT-5.6 Sol: it opens “If you're not sure where to start, use GPT-6 Astra” and features GPT-5.6 Terra and GPT-5.6 Luna after it. Artificial Analysis marks GPT-5.6 Sol deprecated and scores it 46.97 against GPT-6 Sol's 47.53 on v4.3.2. The definition asks what a buyer would choose today, and nobody pays twice as much for the same limits from the same vendor.]
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists this model as eligible, and every OpenAI tier in this catalog is priced from 2026-07-30 or later, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-10. Applied multiplicatively on top of this arrangement discount, which is a derivation: OpenAI states the uplift and the discount separately and never their interaction. Anthropic documents multiplicative composition for the equivalent case, which is why it is the reading taken, and is not a statement about OpenAI.
Reflects the 2026-07-30 cut. OpenAI documents Flex tokens as priced at Batch API rates, and the Batch discount as a flat 50%, so both tiers share this row. Long-context multipliers still apply to the full request. [CORRECTED 2026-08-15: this tier carried a cache-write rate of $2.5/M, 1.25x its input rate, copied from Anthropic’s billing model. OpenAI does not bill a cache write at all — caching is automatic, the first call pays the ordinary input rate and matched prefixes pay the cached rate afterwards. The write bucket is therefore priced at the plain input rate, which is what actually happens. Corroborated by the owner’s own Codex telemetry, where the cache-write counter is structurally zero across 357 sessions and 6.12 billion input tokens.] [REVERSED 2026-08-21, and this is the important half of the note. The "correction" above was wrong: OpenAI does bill a cache write on this model family, at exactly the 1.25x multiplier it was accused of borrowing from Anthropic. Their caching guide states it outright. Under an expander headed “GPT-5.6 and later”: “For GPT-5.6 and later, cache writes cost 1.25× the standard, uncached input-token rate.” Its comparison table says the same in a Cache write charge row, reading “1.25× the uncached input-token rate” for that family against “No additional cache-write charge” for the two older ones.] [CORRECTED 2026-08-27: the sentence quoted here until today was a paraphrase in quotation marks — none of “have no additional fee”, “billed at 1.25” or “reported in cache_write_tokens” appears anywhere on that page. It said the right thing and it was not a quotation, which on a site whose premise is a citable audit trail is its own defect. The same correction removes the absolute figures that followed it — $6.25, $2.50 and $0.25 against inputs of $5.00, $2.00 and $0.20 — because that was the table as it read on 2026-08-21 and Sol was cut on the same day. The ratio is what survived. The telemetry that seemed to corroborate the error is not wrong either; it is pre-GPT-5.6 traffic, on models where the fee genuinely did not exist, and it was generalised to a family that had introduced one on 2026-07-09. A measurement of the wrong thing is more persuasive than no measurement at all, which is exactly why it took a source read to catch.] TOKEN LIMITS, recorded 2026-09-13: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-5.6-terra page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.”
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists this model as eligible, and every OpenAI tier in this catalog is priced from 2026-07-30 or later, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-10. Applied multiplicatively on top of this arrangement discount, which is a derivation: OpenAI states the uplift and the discount separately and never their interaction. Anthropic documents multiplicative composition for the equivalent case, which is why it is the reading taken, and is not a statement about OpenAI.
Reflects the 2026-07-30 cut. Flex and Batch share the published flat 50% multiplier. At $0.10/$0.60 effective, this is the cheapest tier in the catalog by a wide margin. [CORRECTED 2026-08-15: this tier carried a cache-write rate of $0.25/M, 1.25x its input rate, copied from Anthropic’s billing model. OpenAI does not bill a cache write at all — caching is automatic, the first call pays the ordinary input rate and matched prefixes pay the cached rate afterwards. The write bucket is therefore priced at the plain input rate, which is what actually happens. Corroborated by the owner’s own Codex telemetry, where the cache-write counter is structurally zero across 357 sessions and 6.12 billion input tokens.] [REVERSED 2026-08-21, and this is the important half of the note. The "correction" above was wrong: OpenAI does bill a cache write on this model family, at exactly the 1.25x multiplier it was accused of borrowing from Anthropic. Their caching guide states it outright. Under an expander headed “GPT-5.6 and later”: “For GPT-5.6 and later, cache writes cost 1.25× the standard, uncached input-token rate.” Its comparison table says the same in a Cache write charge row, reading “1.25× the uncached input-token rate” for that family against “No additional cache-write charge” for the two older ones.] [CORRECTED 2026-08-27: the sentence quoted here until today was a paraphrase in quotation marks — none of “have no additional fee”, “billed at 1.25” or “reported in cache_write_tokens” appears anywhere on that page. It said the right thing and it was not a quotation, which on a site whose premise is a citable audit trail is its own defect. The same correction removes the absolute figures that followed it — $6.25, $2.50 and $0.25 against inputs of $5.00, $2.00 and $0.20 — because that was the table as it read on 2026-08-21 and Sol was cut on the same day. The ratio is what survived. The telemetry that seemed to corroborate the error is not wrong either; it is pre-GPT-5.6 traffic, on models where the fee genuinely did not exist, and it was generalised to a family that had introduced one on 2026-07-09. A measurement of the wrong thing is more persuasive than no measurement at all, which is exactly why it took a source read to catch.] TOKEN LIMITS, recorded 2026-09-13: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-5.6-luna page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.” [RETIRED 2026-09-25, superseded by GPT-6 Luna, by the lifecycle definition and against OpenAI's current recommendation. OpenAI has neither deprecated GPT-5.6 Luna nor tagged it legacy, and its model index, read the same day, still features it: “Choose GPT-5.6 Terra to balance intelligence and cost, or GPT-5.6 Luna for cost-sensitive, high-volume workloads.” That index features no GPT-6 model but Astra, although GPT-6 Luna was released on 2026-09-22 at half this model's input, cached-input and cache-write rates and $0.50 against $1.20 for output, with the same window, input limit, output cap and long-context step. Artificial Analysis marks GPT-5.6 Luna deprecated and puts the two within a tenth of a point on v4.3.2, 37.32 here against 37.26. For the cost-sensitive, high-volume work OpenAI recommends this model for, a buyer choosing today takes the one at less than half the price, which is the definition's test. What the move gives up is recorded rather than modelled: OpenAI's data-residency table offers gpt-5.6-luna in the United Arab Emirates, where it lists no GPT-6 model, and allows GPT-6 Luna in its European region on Standard processing only.]
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists this model as eligible, and every OpenAI tier in this catalog is priced from 2026-07-30 or later, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-10. Applied multiplicatively on top of this arrangement discount, which is a derivation: OpenAI states the uplift and the discount separately and never their interaction. Anthropic documents multiplicative composition for the equivalent case, which is why it is the reading taken, and is not a statement about OpenAI.
[CORRECTED at the 2026-08-14 refresh: the 2026-10-16 shutdown date this catalog carried is no longer what Google publishes. Its deprecations page now shows the stable gemini-2.5-pro as "No shutdown date announced", so the date is removed rather than left standing as a deadline nobody has committed to.] The entry stays superseded because a newer generation exists and Google points migrations at gemini-3.1-pro-preview — which is itself still Preview, so there is still no GA Pro tier in the Gemini 3 family to move to. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-2.5-pro. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.” STORAGE, recorded 2026-09-26: $4.50 per million tokens for each hour an explicit cache is kept, read from the Standard row of the Gemini 2.5 Pro section of Google's pricing page (last modified 2026-09-24), which prints one storage price for prompts of any length.
Official Batch API and context-caching prices; above 200K the configured 2x input multiplier also yields the published $0.25/M cached rate. The batch discount does not halve the cached-input rate. Superseded because a newer generation exists; the 2026-10-16 shutdown date previously recorded here was removed at the 2026-08-14 refresh after Google's deprecations page reverted to "No shutdown date announced". TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-2.5-pro. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.” STORAGE, recorded 2026-09-26: $4.50 per million tokens for each hour an explicit cache is kept, read from the Batch row of the Gemini 2.5 Pro section of Google's pricing page (last modified 2026-09-24), which prints the same storage price as the Standard row: the batch discount does not reach storage.
Preview (non-GA) tier, and as of 2026-08-01 still the only Gemini 3 Pro tier that exists — Google's own migration path off the retiring 2.5 Pro points here. Above 200K: $4 in / $18 out / $0.40 cached, matching the configured multipliers. Context window confirmed at 1,048,576 input tokens (65,536 output) at the 2026-08-01 refresh, having been omitted rather than guessed previously. Context caching adds a storage fee of $4.50 per million tokens per hour, one rate above 200K and below it, read 2026-09-25. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.1-pro-preview. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
Google's Batch row for gemini-3.1-pro-preview, read 2026-09-26 from the pricing page (last updated 2026-09-24): $1.00 input and $6.00 output per million tokens for prompts up to 200K, and $2.00 and $9.00 above, each half the Standard row's, so this row stores the Standard rates with a batchMultiplier of 0.5. Context caching is printed as “Same as Standard”, $0.20 up to 200K and $0.40 above, so batchCachedInputMultiplier is 1; storage is the Standard row's $4.50 per million tokens per hour, one rate above 200K and below it, so batchCacheStorageMultiplier is 1. Above 200K the configured multipliers give the printed $2.00 in, $9.00 out and $0.40 cached. Flex prints the same rates as Batch and shares this row, and Google's model reference lists the Batch API and Flex inference as supported for this model. Google's Batch API page says a batch request that hits a cache pays “the standard context caching rates”, and this row says the same, so this note carries no CONFLICT paragraph, unlike the six Google batch rows that print a cache price below their Standard row's. Preview (non-GA), like the Standard tier. Since 2026-09-26 the calculator charges storage for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. ADDED 2026-09-26: the catalog carried only the Standard row, so deferrable work on Google's only Gemini 3 Pro model was priced at twice the input and output rates Google charges for it. Google also prints a Batch row for Gemini 3.6 Flash, which is not added: this catalog retired 3.6 Flash for 3.7 Flash, whose Batch tier already charges the identical $0.375 in, $1.875 out, $0.0375 cached and $0.50 storage through 2026-12-31 and the identical list rates after, so a tier added for it would be marked superseded on arrival. TOKEN LIMITS, as on the Standard tier and re-read 2026-09-26: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.1-pro-preview. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
[CORRECTED TWICE. The 2026-08-13 refresh caught that this tier was carried at Google's 2027 list rate, double what is charged, but only the cached figure was actually updated — input and output stayed at $1.50/$7.50 while the note claimed otherwise.] Both are now $0.75/$3.75, matching the live page's "through December 31, 2026" promotional rate. Superseded by Gemini 3.7 Flash, which Google released 2026-08-13 at an identical price and which wins every published benchmark against this tier (DeepSWE 65.3% vs 48.6%, FrontierCode 43.6% vs 34.4%, GDM-MRCR 97.0% vs 91.8%) — there is no workload where paying the same for this one is the better choice. Google has published no shutdown date for it. No long-context tiering. [CORRECTED 2026-09-26: this note said context-cache storage costs $1 per million tokens per hour. Google’s pricing page, read 2026-09-25, prints $0.50 through 2026-12-31, the period this tier covers, and $1.00 from 2027-01-01, which gemini-3-6-flash-list carries; whether it printed $1 when the note was written is not known.] Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.6-flash. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
The rate Google has published as taking effect 2027-01-01, carried separately so a multi-year scenario can be priced against what will actually be charged rather than against a promotion that ends inside the horizon. Exactly double the current promotional rate on every category, cache storage included: $1.00 per 1M tokens per hour against $0.50, read 2026-09-25. Retired 2026-08-15 alongside the promotional 3.6 tier it belongs to: the 3.7 standing rate is identical at $1.50/$7.50 and scores 56 against 52. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.6-flash. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
Released GA on 2026-08-13 and now Google's current Flash tier. Priced identically to 3.6 Flash — $0.75/$3.75/$0.075 through 2026-12-31, doubling on 2027-01-01 — while winning every published benchmark against it, so 3.6 is marked superseded. Token limits of 1,048,576 input and 65,536 output, from Google's API model reference, gemini-3-7-flash-doc; the pricing page states neither. [CORRECTED 2026-09-13: this sentence credited the 1,048,576 to the DeepMind model card, which says only “a token context window of up to 1M”, beside “a 64K token output”. The card's own source record was corrected on 2026-08-29 and this sentence was not.] TOKEN LIMITS, recorded 2026-09-13: maxInputTokens and maxOutputTokens hold the model reference's two limits. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.” No long-context tiering. OMISSION, recorded 2026-08-21: Google bills no cache write, which is why that field is absent — but it does bill cache *storage* by the hour, at $0.50 per 1M tokens rising to $1.00 on 2027-01-01. MODELLED 2026-09-26: this row carries the $0.50, and the list row from 2027-01-01 the $1.00. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. The Gemini 3.1 Pro entry already disclosed its own equivalent; this one did not.
Google publishes Batch and Flex rows for this model at exactly half the standard rate on all three categories — input, output and context caching alike — so a single 0.5 multiplier is the right shape here. It is not the right shape for every Google model: Gemini 2.5 Pro batch halves input and output but charges the same context-caching rate as its standard tier, which is why that entry carries explicit per-category rates instead. Cache storage is not discounted on this row: $0.50 per 1M tokens per hour, as on the Standard row, so batchCacheStorageMultiplier is 1. [CORRECTED 2026-09-26: this sentence said storage is never discounted on any Google batch row, and Gemini 3.1 Flash-Lite’s halves it.] ADDED 2026-08-21. A batch arrangement was published for the catalog's headline Google tier and was not carried, so a reader comparing deferrable work against this model was being shown twice the rate Google charges for it. Flex is priced identically to Batch and shares this row, the same way OpenAI's two are shared. CONFLICT, recorded 2026-09-26: Google's Batch API page, last updated 2026-09-17 and read 2026-09-25, says that a batch request which hits a cache pays “the standard context caching rates”. The pricing page, updated 2026-09-24 and the page the Batch API page sends readers to for details, prints this row's cache price at $0.0375 through 2026-12-31, against $0.075 on the Standard row, and the pricing page's row is what is carried. If the Batch API sentence means the Standard row, a cache hit in a batch costs twice what the calculator charges here. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.7-flash. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
The rate Google publishes as taking effect 2027-01-01, exactly double the current promotional rate on every category. Carried separately so a multi-year horizon can be priced against what will be charged rather than a promotion that ends inside it — and so the "price at standing rates" switch has something to switch to. Whether the increase actually happens is a forecast: Anthropic scheduled a comparable rise for Sonnet 5 and then cancelled it. OMISSION, recorded 2026-08-21: Google bills no cache write, which is why that field is absent — but it does bill cache *storage* by the hour, at $0.50 per 1M tokens rising to $1.00 on 2027-01-01. MODELLED 2026-09-26: this row, which starts on 2027-01-01, carries the $1.00, and the promotional row the $0.50. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. The Gemini 3.1 Pro entry already disclosed its own equivalent; this one did not. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.7-flash. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
Google publishes Batch and Flex rows for this model at exactly half the standard rate on all three categories — input, output and context caching alike — so a single 0.5 multiplier is the right shape here. It is not the right shape for every Google model: Gemini 2.5 Pro batch halves input and output but charges the same context-caching rate as its standard tier, which is why that entry carries explicit per-category rates instead. Cache storage is not discounted on this row: $1.00 per 1M tokens per hour, as on the Standard row, so batchCacheStorageMultiplier is 1. [CORRECTED 2026-09-26: this sentence said storage is never discounted on any Google batch row, and Gemini 3.1 Flash-Lite’s halves it.] ADDED 2026-08-21. A batch arrangement was published for the catalog's headline Google tier and was not carried, so a reader comparing deferrable work against this model was being shown twice the rate Google charges for it. Flex is priced identically to Batch and shares this row, the same way OpenAI's two are shared. This row carries the post-2027 list rate, so the "price at standing rates" switch has somewhere to send a batch scenario as well as a standard one. CONFLICT, recorded 2026-09-26: Google's Batch API page, last updated 2026-09-17 and read 2026-09-25, says that a batch request which hits a cache pays “the standard context caching rates”. The pricing page, updated 2026-09-24 and the page the Batch API page sends readers to for details, prints this row's cache price at $0.075 from 2027-01-01, against $0.15 on the Standard row, and the pricing page's row is what is carried. If the Batch API sentence means the Standard row, a cache hit in a batch costs twice what the calculator charges here. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.7-flash. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
Hit/miss pricing encoded as cached input and uncached input, and the cache write at Moonshot's published five-minute price. Reconfirmed 2026-07-24. [CORRECTED 2026-08-27: this tier carried 1,000,000 tokens of context while the local kimi-k3 entry for the same model carried 1,048,576, so one page of the site contradicted another. Moonshot's own pricing table settles it: the row for kimi-k3 has a Context Window column whose cell reads "1,048,576 tokens" in full. The "1M" that appears in the prose beside it is a pricing unit, not the window — the same page says "Here, 1M = 1,000,000. The prices in the table represent the cost per 1M tokens consumed." Read off the raw HTML rather than a rendered summary, because the figure lives inside the page's own JSX payload. Note this is the opposite outcome from Command A+, where the vendor writes a decimal 200000 in its own config and the decimal reading is therefore the right one: what settles the question is which number the vendor itself spells out, not which is rounder.] OUTPUT LIMIT, read 2026-09-13: none is published below the window. Moonshot's Kimi K3 quickstart says “`max_completion_tokens` defaults to 131072 and can be set up to 1048576.” That ceiling is the window itself, so the window is the only limit recorded, and 131,072 is a default a request can raise rather than a cap. [CORRECTED 2026-09-25: this note began "Hit/miss pricing encoded as cached input and uncached input; the hit/miss model has no separate cache-write fee, so writes are priced at standard input." Moonshot's pricing table now bills the write separately: the kimi-k3 row carries Cache Write Price (TTL 5min) $3.00 and Cache Write Price (TTL 1h) $6.00 beside the cached, input and output prices, the page says "For K3 series models, cache writes are billed separately per TTL tier (5min / 1h)", and the five-minute tier is the default when a request names none. No figure moves. The stored $3.00 is now Moonshot's own write price for the default tier rather than the input rate by assumption.] ONE-HOUR WRITE, recorded 2026-09-25: the $6.00 in the same row is carried beside it, twice the five-minute price, and which of the two a write pays is the scenario's one-hour share.
Released 2026-08-12 on the same 1.5T foundation as Grok 4.5 — xAI attributes the gains to longer post-training, not a larger model. Input and output are unchanged from 4.5, but cached input rose from $0.30 to $0.50, so a cache-heavy workload is the one case where the newer model costs more. Uniform 2x above 200K on input, cached input and output. No batch discount is published for this tier (xAI lists one only for the Grok 4.3 and 4.20 lines). [CORRECTED 2026-08-21: the long-context boundary is inclusive here and the engine treated it as strict, so a prompt of exactly 200,000 tokens was billed at the single rate. xAI: "requests whose prompt reaches the listed token threshold are billed at the higher rate for all tokens in the request", which is also what this tier's own label says. Also recorded, because the note previously said only that no batch discount is published: xAI states that "grok-4.6 and grok-4.5 are not currently supported for Batch API requests and will be rejected". Batch is unavailable here, not merely undiscounted. Where xAI does offer it the discount is 20% rather than 50%, applying to input, output, cached and reasoning tokens alike.] OUTPUT LIMIT, read 2026-09-13: xAI's Grok 4.6 guide gives the output limit as “No text output limit” and the context window as 500,000 tokens, so the window is the only limit recorded. RE-CITED 2026-09-25: xAI's Grok 4.6 guide, cited above for the output limit and the window, now redirects to the Grok 4.7 guide. Both facts stand and are cited where they are still printed for this model: the release notes' 2026-08-12 entry says Grok 4.6 has "a 500k context window" and "no text output limit", and the grok-4.6 model page gives the window as 500,000 and Batch API as "Not supported". Prices re-read the same day on the pricing page and the model page, unchanged. [RETIRED 2026-09-25, superseded by Grok 4.7. xAI still lists grok-4.6 on its pricing page at these rates and has announced no deprecation. But the address of its Grok 4.6 guide now redirects to the Grok 4.7 guide, and xAI's announcement of 2026-09-21 says Grok 4.7 is “Served at the same price and speed as Grok 4.6” and “uses a new, larger base model compared to Grok 4.6”; the comparison table in it puts Grok 4.7 ahead of Grok 4.6 on all seven benchmarks it prints. At an identical price a buyer would choose Grok 4.7 today, which is the lifecycle definition's test. Grok 4.5 was retired in favour of this model on the same ground, with no deprecation tag either.]
xAI states: “Token usage on the US endpoint costs 10% more than on the global endpoint. The premium applies to input, output, and cached input tokens, including long-context rates, and prompt caching discounts still apply.” The same page names this model as one of the two the US endpoint serves: “The US endpoint currently serves two models, grok-4.7 and grok-4.6.” xAI publishes no cache-write price, so the engine bills a write as input, and the premium covers input. Global is the default and is what every rate in this entry is. Read 2026-09-25.
Grok 4.6 (xhigh), the highest of the four efforts Artificial Analysis lists, and the one Grok 4.7 is scored at: the page prints “scores 44”. [CORRECTED 2026-09-26: this cited the page at /models/grok-4-6, which is Grok 4.6 (high), at 44.31 on 2026-09-25 and 2026-09-26 against 44.20 here, so the score shown does not change.]
Superseded as flagship by Grok 4.6 on 2026-08-12, but still fully listed and priced by xAI with no deprecation tag — and its $0.30 cached input is cheaper than its successor's $0.50. Uniform 2x above the 200K threshold. Note the 500K window is smaller than the older Grok 4.3 line's 1M. [CORRECTED 2026-08-21: the long-context boundary is inclusive here and the engine treated it as strict, so a prompt of exactly 200,000 tokens was billed at the single rate. xAI: "requests whose prompt reaches the listed token threshold are billed at the higher rate for all tokens in the request", which is also what this tier's own label says. Also recorded, because the note previously said only that no batch discount is published: xAI states that "grok-4.6 and grok-4.5 are not currently supported for Batch API requests and will be rejected". Batch is unavailable here, not merely undiscounted. Where xAI does offer it the discount is 20% rather than 50%, applying to input, output, cached and reasoning tokens alike.] REPOINTED 2026-09-25: its replacement moves from Grok 4.6 to Grok 4.7, because Grok 4.6 is now retired itself and a replacement has to be a live tier, or the calculator's switch to the current equivalent would land on another retired one.
xAI: “grok-4.7 and grok-4.6 are currently the only models available on the US endpoint”, and “Requesting a model that is not on that list, including grok-latest, fails with 404 Not Found.” So this tier cannot be pinned to the United States at any price, which is a different answer from a surcharge nobody has read. Read 2026-09-25.
Meta's frontier model is closed-weight and, since 2026-08-05, carries a published per-token price — the first time this catalog can compare against Meta's best model rather than only its open derivatives. Meta said on 2026-08-10 that it intends to open these weights "in the coming weeks"; that had not happened by the 2026-08-13 cutoff. No batch tier or long-context surcharge is published. [CORRECTED 2026-08-27: context was recorded as 1,000,000. Meta's own models page states 1,048,576 for every Muse Spark ID, in a table cell and again in prose. The old figure understated the window by 4.6% on the field that decides whether a request is too large to send, so it was conservative rather than dangerous — but it was also the reason two pages of this site disagreed about the same model.]
An opt-in tier at roughly a twentieth of the standard rate, priced on the condition that Meta trains future models on the requests. Carried because it is the cheapest frontier-lab tier in the catalog by a wide margin and therefore the hardest comparator any self-hosting case has to beat — but the calculator flags it, since data control is one of the main reasons to own hardware and this tier trades exactly that away. [CORRECTED 2026-08-27: context was recorded as 1,000,000. Meta's own models page states 1,048,576 for every Muse Spark ID, in a table cell and again in prose. The old figure understated the window by 4.6% on the field that decides whether a request is too large to send, so it was conservative rather than dangerous — but it was also the reason two pages of this site disagreed about the same model.]
CACHE MODE, read 2026-10-07 from the Singapore column of the model's Model Studio page and from the context cache guide: stored under implicit caching, Alibaba's automatic prefix caching. A cache hit bills $0.25 per million tokens and a cache write bills the input rate, $2.00, because implicit caching charges no creation fee. Explicit caching would instead create an entry at $2.50 per million, 125% of input, and read it at $0.17, for entries that live 5 minutes unless a hit renews them; this catalog stores no Alibaba tier that way. Implicit hits are not guaranteed, and a prefix needs at least 1,024 tokens to be eligible. [REVERSED 2026-10-07: this tier stored the explicit creation price, $2.50, as its write price beside the implicit hit of $0.25, the pairing the correction of 2026-08-21 below introduced. No request is billed at that pair: at 70% hits and 5% writes it cost $0.800 per million input tokens, against $0.775 under implicit caching and $0.744 under explicit caching.] The 2.4T flagship previewed at WAIC on 2026-07-19 now has a published per-token price, so it is no longer excluded for want of one. Rates are the Singapore/international region; Alibaba's Beijing region prices it lower ($1.65 / $4.951) and this catalog records the international figures throughout. Max input is 991,808 tokens with 131,072 output inside the stated 1M window. [CORRECTED 2026-10-07: this said “The consolidated Model Studio pricing table did not yet carry this row at the cutoff, so the per-model documentation page is the source; confidence is medium accordingly.” The consolidated table carries the row now, $2 input and $6 output, read 2026-10-07, and confidence is high.] [CORRECTED 2026-08-21: no cache-write price was recorded, so the engine fell back to the plain input rate of $2.00. Alibaba publishes "Explicit Cache Creation | 2.5 | Per 1M tokens" for the Singapore region — 125% of input, 25% above the fallback.] Worth knowing that Alibaba documents three cache prices rather than two: explicit creation at 125% of input, an implicit hit at 12.5% and an explicit read at 8.5%. The figure recorded here as cachedInput is the printed implicit-hit one, $0.25; the explicit read is $0.17. [CORRECTED 2026-08-29: two of those three percentages read 10% and 20%, which are neither the stored figures nor Alibaba's. Re-read from the same page, the Singapore table prints Input 2, Output 6, Input(Implicit Cache) 0.25, Explicit Cache Creation 2.5 and Explicit Cache Read 0.17, so the ratios are 12.5% and 8.5%; the Beijing column gives 0.206, 2.063 and 0.137 against an input of 1.65, which is the same three ratios. Every stored dollar figure was right and only the prose was wrong, which is the harder version of this defect to notice.] SNAPSHOT 0902, read 2026-09-08: a dated build exists as API id qwen3.8-max-0902 (alias qwen3.8-max-2026-09-02), post-trained for engineering-scale projects and multi-tool agent orchestration. Alibaba and OpenRouter both state its pricing is unchanged from the base alias — same $2/$6, same three-tier cache pricing, same 1,000,000-token window — so it is named here rather than carried as a second row of identical numbers. Its release date is unresolved: TechNode's article is dated 2026-09-02 and OpenRouter's model page says September 3; no Alibaba-dated announcement was found to break the tie, and the two are recorded rather than averaged. The international price is also not printed in dollars on Alibaba's own page: that table renders in CNY (14.988 / 44.965 / 1.874 / 18.736 / 1.274 per 1M), and every one of those five divides by the same ~7.495 into the USD figures here, which OpenRouter states directly. TOKEN LIMITS, recorded 2026-09-13: maxInputTokens 991,808 and maxOutputTokens 131,072, re-read from the Context Limits table on Alibaba's qwen3.8-max page, which prints Max Input Length 991808, Max Output Length 131072 and Context Window 1000000. The same table gives thinking mode a lower Max Input Length of 983616, which a scenario cannot select and is not stored, so a thinking-mode request with more than 983,616 input tokens, up to 991,808, passes here although Alibaba refuses it. Whether reasoning counts toward the output length is not stated and not recorded; the table's Max Chain-of-Thought Length of 262144 is larger than the output length, which suggests reasoning is counted apart from it. RE-READ 2026-09-25, and two things this note says above no longer hold. Alibaba's consolidated Model Studio pricing table now carries qwen3.8-max and qwen3.8-max-0902 at $2 input and $6 output for the Singapore region, which is the row whose absence set this entry's confidence at medium; and the per-model page now prints the Singapore column in dollars — Input 2, Output 6, Input(Implicit Cache) 0.25, Explicit Cache Creation 2.5, Explicit Cache Read 0.17 — where on 2026-09-08 it rendered them in CNY. Two of Alibaba's own pages now state every figure here, so confidence is high. The same page prices the model lower in its Global deployment scope, offered from Frankfurt, Virginia, Tokyo and Hong Kong: $1.65 input, $4.951 output and the Beijing column's cache prices. This catalog records the International scope throughout, which is the Singapore column.
Hit/miss pricing: cache hits $0.0028/M, misses at standard input; no separate write fee, so writes are encoded at standard input. This flat rate ended at 16:00 UTC on 2026-08-16, replaced by peak/off-peak billing that is more expensive at every hour — see the successor tier. [RETIRED 2026-08-21: the end date is now behind the research cutoff, so the tier is marked superseded. It stays resolvable, because scenario URLs shared while it was live must keep opening on the price they were shared at.] REPOINTED 2026-09-11: supersededBy named deepseek-v4-flash-api-tod, which has itself now been retired behind DeepSeek-V4.1-Flash. A chain of retired entries leaves the UI offer to switch to the replacement pointing at something equally retired, so this now names the live tier directly.
Hit/miss pricing: cache hits $0.003625/M, misses at standard input; no separate write fee, so writes are encoded at standard input. This flat rate ended at 16:00 UTC on 2026-08-16, replaced by peak/off-peak billing that is more expensive at every hour — see the successor tier. [RETIRED 2026-08-21: the end date is now behind the research cutoff, so the tier is marked superseded. It stays resolvable, because scenario URLs shared while it was live must keep opening on the price they were shared at.]
Effective 16:00 UTC on 2026-08-16. DeepSeek: "Peak hours are 01:00 - 04:00 and 06:00 - 10:00 UTC, Monday through Friday (all other hours are off-peak)." [CORRECTED 2026-08-27: this note previously quoted that sentence without "Monday through Friday" and derived 7 peak hours against 17 off-peak, which reads as 29.2% of the week at the peak rate. Weekends are entirely off-peak, so the real split is 35 peak hours of 168 (20.8%) against 133 off-peak (79.2%). Peak exposure therefore falls by two sevenths. The cost effect is much smaller than that sounds and is worth stating separately, because off-peak is half of peak rather than free: the blended multiplier moves from 0.7083x0.5 + 0.2917 = 0.6458 to 0.7917x0.5 + 0.2083 = 0.6042, so a workload spread evenly over the week costs 6.5% less, not 29% less. An earlier version of this note said 29%, which confused a change in exposure with a change in price. The clipped clause was the load-bearing one: everything else in the sentence is a schedule, and that phrase is what turns it into a weekly fraction.] Listed figures are the PEAK rates; off-peak is exactly half on all three categories. READ THIS BEFORE TREATING IT AS A DISCOUNT: off-peak is not a discount off today's price, it is a discount off a newly raised one. Against the flat rate it replaces, off-peak is 1.6x on cache-miss input, 2.4x on output and 2.5x on cache hits; peak is 3.1x, 4.7x and 5.0x. The whole rate card went up several-fold and the peak/off-peak split is layered on top. [RETIRED 2026-09-11: the pricing page now states that the legacy names deepseek-v4-flash and deepseek-v4-flash-vision-exp are still accepted but the corresponding models have been retired, their requests served by DeepSeek-V4.1-Flash and billed at the Flash price. The endpoint still answers, so this is not a 404 — it is the same name in front of a different model at a different price, which is the case a superseded tier exists for. It stays resolvable, because scenario URLs shared while it was live must keep opening on the price they were shared at.]
Effective 16:00 UTC on 2026-08-16, alongside the V4 Pro GA release. Listed figures are the PEAK rates; off-peak is exactly half on all three categories. The increase is steepest here: against the flat rate it replaces, off-peak is 1.5x on cache-miss input, 2.3x on output and 6.1x on cache hits, and peak reaches 3.0x, 4.6x and 12.1x. A cache-heavy agentic workload is hit hardest, since the cache-hit rate rose furthest. The peak/off-peak schedule is the same one quoted on the V4-Flash time-of-day tier, weekdays only; that entry carries the vendor's sentence and the weekly split. CONFLICT, recorded 2026-09-13 and deliberately left unresolved: DeepSeek's own pages disagree about this endpoint from 2026-09-14. The V4.1-Flash release note, dated 2026-09-10, says "Starting at 04:00 UTC on Sept 14, 2026, all deepseek-v4-pro requests will route to V4.1-Flash at V4.1-Flash rates. This will continue until V4.1-Pro launches." The pricing page, read on 2026-09-13, footnotes the deepseek-v4-pro column with "In response to user demand, we have decided to continue providing API services for DeepSeek V4 Pro after September 14, 2026, with the billing method remaining unchanged." The footnote carries no date of its own, and neither page mentions the other. The tier stays current at the V4 Pro rates, which is what the endpoint bills until 2026-09-14 on both readings and after it on the pricing page's. No effectiveUntil is set, because that field would record the release note's date as the end of this rate and settle the conflict in the release note's favour. If the routing does take effect, a deepseek-v4-pro request is billed at the rates on deepseek-v4-1-flash-api-tod, which is a different model. Revisit after 2026-09-14. OUTPUT CAP, recorded 2026-09-13: 393,216 tokens. The pricing page prints MAX OUTPUT as “MAXIMUM: 384K” in a row shared by both model columns, and DeepSeek's chat completion reference spells it out: max_tokens “must be between 1 and 384K (393216)”. That reference does not say whether thinking-mode reasoning counts toward max_tokens, so that is not recorded, and the pricing page publishes no input limit below the window. RESOLVED 2026-09-25, in the pricing page's favour: V4 Pro is still served, at these rates. Eleven days after the routing date, the pricing page still lists deepseek-v4-pro as model version DeepSeek-V4-Pro-0813 at $1.32 cache-miss input, $0.044 cache hit and $3.96 output at peak, with its own concurrency limit of 500. The footnote quoted above has left the pricing page, and the same sentence now sits in DeepSeek's change log, in its 2026-09-10 entry: "In response to user demand, we have decided to continue providing API services for DeepSeek V4 Pro after September 14, 2026, with the billing method remaining unchanged. We will provide further notice should there be any changes." Only the release note still says the requests route to V4.1-Flash, and it still says "We're phasing out V4-Pro", so a retirement is to be expected, with notice. HOLIDAYS, recorded 2026-09-25: the pricing page's footnote now reads "Off-peak rates are half of the peak rates. Peak hours are 01:00 - 04:00 and 06:00 - 10:00 UTC, Monday through Friday, excluding Chinese public holidays. All other hours are off-peak, including weekends and Chinese public holidays in full." The same sentence as deepseek-v4-flash-api-tod quotes it, read on 2026-08-27, named no holidays. The schedule this tier carries cannot express a holiday calendar, so a weekday inside a Chinese public holiday is costed at its peak hours like any other weekday. That overstates the peak share, and so the cost, by as many such weekdays as a horizon holds; DeepSeek names no calendar, and none is modelled here.
CACHE MODE, read 2026-10-07 from the Singapore column of the model's Model Studio page and from the context cache guide: stored under implicit caching, Alibaba's automatic prefix caching. A cache hit bills $0.50 per million tokens and a cache write bills the input rate, $2.50, because implicit caching charges no creation fee. Explicit caching would instead create an entry at $3.125 per million, 125% of input, and read it at $0.25, for entries that live 5 minutes unless a hit renews them; this catalog stores no Alibaba tier that way. Implicit hits are not guaranteed, and a prefix needs at least 1,024 tokens to be eligible. [REVERSED 2026-10-07: this tier stored the explicit pair, a read of $0.25 with a creation price of $3.125, where Qwen3.8-Max, Qwen3.8-Flash and Qwen3.7-Plus stored the implicit hit beside the explicit creation price, so the tiers of one provider were compared on different assumptions. At 70% hits and 5% writes the explicit pair billed $0.956 per million input tokens against $1.100 under implicit caching, which understated the tier by 13%.] CONFLICT RESOLVED at the 2026-08-01 refresh. The earlier reading — that "$2.5, limited-time 50% off" implied a $5/$15 list price — was wrong: the list price on Alibaba's page is itself $2.50 in / $7.50 out, and the 50% discount applies on top of it, giving an effective $1.25/$3.75 that matches the third-party trackers. The discount badge appears only on the mutable `qwen3.7-max` alias; pinned dated snapshots such as `qwen3.7-max-2026-06-08` bill the plain $2.50/$7.50. This entry records the undiscounted list rate, which is what a pinned model id costs and what remains once the promotion ends. [CORRECTED 2026-10-07: this read "Cache hits bill at 10% of input and cache writes at 125%, both encoded.", which are Alibaba's explicit-caching terms for this model: a read of $0.25 and a creation price of $3.125. The tier now stores its implicit-caching terms, as the first paragraph says.] Context window is not stated on the pricing page, and was omitted rather than guessed. ADDED 2026-09-13: Alibaba's Model Studio page for qwen3.7-max prints a Context Limits table, Max Input Length 991808, Max Output Length 131072 and Context Window 1000000, and all three are now recorded. The same table gives thinking mode a lower Max Input Length of 983616, which a scenario cannot select and is not stored, so a thinking-mode request with more than 983,616 input tokens, up to 991,808, passes here although Alibaba refuses it. Whether reasoning counts toward the output length is not stated and not recorded; the table's Max Chain-of-Thought Length of 262144 is larger than the output length, which suggests reasoning is counted apart from it. RE-READ 2026-09-25: the consolidated table lists qwen3.7-max, qwen3.7-max-2026-06-08 and qwen3.7-max-2026-05-20 at $2.5 input and $7.5 output, and none of the three rows carries a promotion badge any more, so the discount described above has ended and the recorded list rate is what the mutable alias bills as well.
CACHE MODE, read 2026-10-07 from the Singapore column of the model's Model Studio page and from the context cache guide: stored under implicit caching, Alibaba's automatic prefix caching. A cache hit bills $0.08 per million tokens and a cache write bills the input rate, $0.40, because implicit caching charges no creation fee. Explicit caching would instead create an entry at $0.50 per million, 125% of input, and read it at $0.04, for entries that live 5 minutes unless a hit renews them; this catalog stores no Alibaba tier that way. Implicit hits are not guaranteed, and a prefix needs at least 1,024 tokens to be eligible. Above 256K input the page prints an implicit hit of $0.24 against an input of $1.20, three times each of the figures here, which the long-context multipliers reproduce. [REVERSED 2026-10-07: this tier stored the implicit hit of $0.08 beside the explicit creation price of $0.50, a pair no request is billed at. At 70% hits and 5% writes it cost $0.181 per million input tokens, against $0.176 under implicit caching and $0.153 under explicit caching.] Tiered pricing verified 2026-07-24: input $0.40 to 256K and $1.20 above, output $1.60 to 256K and $4.80 above (both 3x). Context window is not stated on the pricing page, and was omitted rather than guessed. ADDED 2026-09-13: Alibaba's Model Studio page for qwen3.7-plus prints a Context Limits table, Max Input Length 991808, Max Output Length 131072 and Context Window 1000000, and all three are now recorded. The same table gives thinking mode a lower Max Input Length of 983616, which a scenario cannot select and is not stored, so a thinking-mode request with more than 983,616 input tokens, up to 991,808, passes here although Alibaba refuses it. Whether reasoning counts toward the output length is not stated and not recorded; the table's Max Chain-of-Thought Length of 262144 is larger than the output length, which suggests reasoning is counted apart from it. RE-READ 2026-09-25: unchanged. The consolidated table still reads "List price $0.4 (Limited-time 20% off)" and "List price $1.6" on the mutable qwen3.7-plus alias and plain $0.4 and $1.6 on the pinned qwen3.7-plus-2026-05-26, tripling above 256K; the list rate is what is recorded. CACHE PRICES ADDED, read 2026-09-25 from the same page: the Singapore table prints Input(Implicit Cache) 0.08, Explicit Cache Creation 0.5 and Explicit Cache Read 0.04 up to 256K, and 0.24, 1.5 and 0.12 from 256K to 1M, three times as much like every other row, which the long-context multipliers here already apply to both cache prices. Until then no cache price was recorded since the 2026-07-24 read, so a cache hit was billed at the full $0.40 input rate and a write at the same, each with a caution calling its price unpublished. [CORRECTED 2026-10-07: this read "Recorded as the two Qwen3.8 tiers record theirs: the implicit hit, 20% of input, as cachedInput, and explicit creation, 125%, as the write price. That pair is no single billing mode, since implicit caching charges no creation fee and explicit caching reads at the $0.04 printed for it, and Qwen3.7-Max stores its explicit pair instead; docs/BUGS.md records the choice left open." The choice was made on 2026-10-07: every Alibaba tier is stored under implicit caching, with the write billed at the input rate.]
Overseas USD pricing; digits identical to DeepSeek V4 Flash on both tiers — apparent deliberate price-matching, verified on both primary pages 2026-07-24. Context from the model card; the pricing page does not state windows. Hit/miss model; writes at standard input. [CORRECTED 2026-08-21: recorded as a hit/miss tier with writes priced at the standard input rate. Xiaomi does publish a distinct cache-write line, and it currently reads "Cache Write: Limited-time Free" — so the write bucket costs nothing here. Carried as zero with the caveat that it is a promotion with no published end date, which is a different kind of zero from OpenAI’s pre-5.6 structural one: this can be withdrawn without the model changing.] OUTPUT CAP, recorded 2026-09-13: 131,072 tokens, reasoning included. Xiaomi's MiMo-V2.5 model page lists a max output of 128K tokens, and its chat API reference gives max_completion_tokens a required range of [1, 131072] and describes it as “An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.” [RETIRED 2026-09-25, superseded by MiMo V2.6 Flash. Xiaomi's pricing page, updated 2026-09-22, marks this model “(to be deprecated)” and says “mimo-v2.5-pro and mimo-v2.5 will be officially deprecated at 10:00 (Beijing time) on October 21,2026. It is recommended to switch to the new version of the models as soon as possible.” Its deprecation page gives this model the note “No system replacement model; will be directly deprecated upon expiration”, so from that hour a request naming it fails rather than being rerouted. MiMo V2.6 Flash is priced on the same row, and Xiaomi says it “has comprehensively outperformed MiMo-V2.5-Pro”, the stronger of the two models it replaces, so it is what a buyer would choose today, which is the lifecycle definition's test. The shutdown date is recorded here and not as an effectiveUntil, which the calculator reads as a price that changes within the same model and would report as ending with no successor.]
Artificial Analysis's own estimate, not its measurement: the model page prints “scores 25 (estimated)” and keys the figure “Estimate (independent evaluation forthcoming)”.
Cached input printed as $0.0036/M on MiMo's pricing page versus DeepSeek V4 Pro's $0.003625/M for the equivalent tier — a small cross-vendor print difference, preserved rather than reconciled. Context from the model card; the pricing page does not state windows. Hit/miss model; writes at standard input. [CORRECTED 2026-08-21: recorded as a hit/miss tier with writes priced at the standard input rate. Xiaomi does publish a distinct cache-write line, and it currently reads "Cache Write: Limited-time Free" — so the write bucket costs nothing here. Carried as zero with the caveat that it is a promotion with no published end date, which is a different kind of zero from OpenAI’s pre-5.6 structural one: this can be withdrawn without the model changing.] OUTPUT CAP, recorded 2026-09-13: 131,072 tokens, reasoning included. Xiaomi's MiMo-V2.5-Pro model page lists a max output of 128K tokens, and its chat API reference gives max_completion_tokens a required range of [1, 131072] and describes it as “An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.” [RETIRED 2026-09-25, superseded by MiMo V2.6 Pro. Xiaomi's pricing page, updated 2026-09-22, marks this model “(to be deprecated)” and says “mimo-v2.5-pro and mimo-v2.5 will be officially deprecated at 10:00 (Beijing time) on October 21,2026. It is recommended to switch to the new version of the models as soon as possible.” Its deprecation page gives this model the note “No system replacement model; will be directly deprecated upon expiration”, so from that hour a request naming it fails rather than being rerouted. MiMo V2.6 Pro is priced on the same row, adds a Batch rate at half, and is the version Xiaomi tells its users to switch to, so it is what a buyer would choose today, which is the lifecycle definition's test. The shutdown date is recorded here and not as an effectiveUntil, which the calculator reads as a price that changes within the same model and would report as ending with no successor.]
Current proprietary flagship (v26.04); also published as open weights (Modified MIT). Generic 90% cached-input discount; no cache-write price published. TOKEN LIMITS, read 2026-09-13: no input limit or output cap separate from the window is recorded. Mistral's model page gives Mistral Medium 3.5's context as 256k, and its chat completion reference bounds a request only by that window: “The token count of your prompt plus max_tokens cannot exceed the model's context length.” The 256,000 stored is the decimal reading of 256k, because no page cited here spells this window out in digits. Mistral Large 3's card says 256k as well, and this catalog stores 262,144 for that model because the vLLM launch command on that card passes --max-model-len 262144; the card cited here carries no launch command. Under the binary reading, 262,144, a request of 256,001 to 262,144 tokens would be accepted by Mistral and is refused here.
The multiplier and its scope are quoted: “Regional inference is billed at 1.1× standard list pricing (a 10% upcharge) for input tokens, output tokens, cached reads, and cache writes.” That this model is served in a region is a reading, and graded as one. The same page says “Available models vary by region” and lists none. The pricing page offers a “Regional inference +10%” switch “for supported models”, and its markup attaches that switch to both of this model’s prices, which is the evidence taken. A regional models.list call would settle it and needs an API key. Read 2026-09-25.
Mistral publishes a generic 50% Batch discount and, separately, that cached input tokens reduce input cost by up to 90%. It never says whether the two stack, and publishes no per-model batch prices to derive it from. [DOWNGRADED 2026-08-21 from high to medium confidence. Applying the batch multiplier to the cached bucket as well as the plain one is this catalog's reading, not a published rate — and it is a reading the other providers do not settle either way: Anthropic and OpenAI do halve every category, Google halves context caching on Gemini 3.7 Flash but not on 2.5 Pro, and xAI discounts 20% rather than 50%. A field graded high confidence on an unstated figure is the failure mode this catalog exists to avoid.] REGRADED 2026-08-29: the 2026-08-21 pass moved confidence to medium and left evidenceKind at official-claim. Confidence and evidence kind are different axes — a reading is not the vendor's claim at any confidence — so this is now estimated. CACHE HITS IN BATCH, recorded 2026-09-25: the discount on cache hits is now recorded as not stated (null) instead of riding on the 0.5 unmarked. The calculator still charges it — $0.075 per million, the $0.15 cache-hit price halved again, which is the lower of the two readings — and its diagnostics give what the other reading, $0.15, would add; the price table shows both. Mistral’s page says batch runs “at half price” and that cached input tokens are “cutting input costs by 90%”, and nothing about the two together. The configurator on that page multiplies them: the script it loaded defines batch as 0.5 and cache as 0.1 and applies every option selected, so with both selected it computes $0.075 and, rounding any base price of $1 or more to the cent, shows $0.08. That is a display, not a billing statement, and the tier stays estimated because the price the calculator charges a cache hit here rests on it. TOKEN LIMITS, read 2026-09-13: no input limit or output cap separate from the window is recorded. Mistral's model page gives Mistral Medium 3.5's context as 256k, and its chat completion reference bounds a request only by that window: “The token count of your prompt plus max_tokens cannot exceed the model's context length.” The 256,000 stored is the decimal reading of 256k, because no page cited here spells this window out in digits. Mistral Large 3's card says 256k as well, and this catalog stores 262,144 for that model because the vLLM launch command on that card passes --max-model-len 262144; the card cited here carries no launch command. Under the binary reading, 262,144, a request of 256,001 to 262,144 tokens would be accepted by Mistral and is refused here.
Mistral: “Stateful features, including Agents, Batch, and the Files API, are not available on regional endpoints.” So the batch arrangement this tier prices cannot be pinned to a jurisdiction, while the standard tier can. Read 2026-09-25.
Z.ai publishes this now, and the assumption it replaces was exact. Their pricing table reads "GLM-5.3 | $1.4 | $0.26 | Limited-time Free | $4.4" under the headings Input, Cached Input, Cached Input Storage and Output — identical to the GLM-5.2 row directly beneath it, which is what this tier had assumed on the vendor's same-base-model statement. [UPGRADED 2026-08-21 from an assumption at low confidence. The previous note read "NOT A PUBLISHED PRICE" and described a model reachable only through a coding plan with no credits-per-token rate; that was true when written and is no longer. The id still says "assumed" because a scenario URL shared while it was one has to keep resolving — an id is a contract, not a description.] Worth noting what Z.ai's third column is: a cache *storage* charge, currently waived, not a cache-write fee. There is no write line here, so the write bucket is priced at the plain input rate as it is for the other hit/miss providers — a convention of this catalog rather than a published fee. OUTPUT CAP, recorded 2026-09-13: 131,072 tokens. Z.ai's GLM-5.3 guide gives “a maximum output length of 128K tokens”, and its chat completion reference gives max_tokens a maximum of 131072 under the description “The maximum number of tokens for model output, the GLM-5.3, GLM-5.2, GLM-5.1, GLM-5, GLM-4.7, GLM-4.6 series supports 128K maximum output”. Neither page says whether reasoning counts toward max_tokens, so that is not recorded.
Context window from Z.ai's GLM-5.2 guide, which gives it as 1M beside a maximum output of 128K. [CORRECTED 2026-09-13: this sentence read “Context window confirmed at 1,048,576 tokens (128K max output) on Z.ai's own documentation at the 2026-08-01 refresh, having previously rested on secondary sources only” and cited no documentation page, and the guide, read on 2026-09-13, prints neither figure in digits. Z.ai spells out only the output figure: its chat completion reference gives the GLM-5.2 series “128K maximum output” with a maximum of 131072. The 1,048,576 applies the same binary reading to 1M, which Z.ai does not spell out.] OUTPUT CAP, recorded 2026-09-13: maxOutputTokens holds that 131,072. Neither page says whether reasoning counts toward max_tokens, so that is not recorded. No long-context tiering — one flat rate across the window. Z.ai publishes no batch API, so there is no batch row to encode; cache storage is currently free as a limited-time promotion.
Z.ai's pricing page carries this row with the list prices struck through: <del>$0.15</del> $0.075 input, <del>$0.03</del> $0.015 cached input, <del>$0.50</del> $0.25 output, and "GLM-5.3-Flash is available at a 50% discount (strikethrough prices are list prices). The promotion ends at 24:00 on September 9, 2026 (UTC+8, Singapore time)." A first extraction of the same page returned only the discounted figures with no sign a promotion existed, because the markup was dropped; the row was re-read from raw HTML to confirm. The standing rate is carried separately as glm-5-3-flash-api-list. No cache-write line is published, so the write bucket falls back to the plain input rate. Cached input storage is free for the same limited period. GLM-5.3 and GLM-5.2 carry no strikethrough and are unchanged. OUTPUT CAP, recorded 2026-09-13: 131,072 tokens. Z.ai's GLM-5.3-Flash guide lists Maximum Output Tokens as 128K, and its chat completion reference says “`GLM-5.3-Flash` supports a maximum output length of 128K” and gives max_tokens a maximum of 131072. Neither page says whether reasoning counts toward max_tokens, so that is not recorded. [RETIRED 2026-09-14: the promotion ended at 24:00 on 2026-09-09 (UTC+8), which is behind the 2026-09-13 research cutoff, so the tier is marked superseded and points at glm-5-3-flash-api-list, the list rate that applies from 2026-09-10. The end date is the one Z.ai published and quoted above, so the retirement rests on no reading of the page after the cutoff. It stays resolvable, because scenario URLs shared while it was live must keep opening on the price they were shared at.]
The struck-through figures on Z.ai's own pricing row, which the same page names as the list prices the 50% promotion is measured against. Carried as its own tier so a horizon that outlives 2026-09-09 can be costed against the rate that will actually apply, rather than silently keeping a lapsed discount. OUTPUT CAP, recorded 2026-09-13: 131,072 tokens. Z.ai's GLM-5.3-Flash guide lists Maximum Output Tokens as 128K, and its chat completion reference says “`GLM-5.3-Flash` supports a maximum output length of 128K” and gives max_tokens a maximum of 131072. Neither page says whether reasoning counts toward max_tokens, so that is not recorded. RE-READ 2026-09-25: unchanged at $0.15 / $0.03 / $0.50. The table now also lists GLM-5.3-FlashX at $0.37 input, $0.075 cached input and $1.25 output. Z.ai's guide for this model is now titled “GLM-5.3-Flash/FlashX”, gives the two model codes glm-5.3-flash and glm-5.3-flashx side by side, and says only that FlashX delivers “inference speeds of 200 tokens/s”; it names no other difference. It is recorded here rather than carried as a tier: it would be a second standard-rate tier for one model, at about 2.5 times this one's price for speed alone, and the calculator costs tokens, not latency.
CACHE MODE, read 2026-10-07 from the Singapore column of the model's Model Studio page and from the context cache guide: stored under implicit caching, Alibaba's automatic prefix caching. A cache hit bills $0.016 per million tokens and a cache write bills the input rate, $0.15, because implicit caching charges no creation fee. Explicit caching would instead create an entry at $0.20 per million, 133% of input, and read it at $0.016, the same price as an implicit hit, for entries that live 5 minutes unless a hit renews them; this catalog stores no Alibaba tier that way. Implicit hits are not guaranteed, and a prefix needs at least 1,024 tokens to be eligible. [REVERSED 2026-10-07: this tier, labelled "standard · explicit cache", stored the implicit hit of $0.016 beside the explicit creation price of $0.20. Because explicit caching reads at the same $0.016, that was the explicit pair, and at 70% hits and 5% writes it cost $0.0587 per million input tokens against $0.0562 under implicit caching.] Qwen Cloud's own model page: Input $0.15, Input (Implicit Cache) $0.016, Explicit Cache Creation $0.2, Explicit Cache Read $0.016, Output $0.47 per 1M tokens. The figure recorded as cachedInput is the printed implicit-hit rate, following the convention already documented on qwen3-8-max. Graded medium confidence because Qwen Cloud was the only primary source for these prices when they were recorded: Alibaba Model Studio's consolidated pricing table carried no qwen3.8-flash row, and its per-model page 302-redirected to a not-found page. That page resolves at the same address as of 2026-09-13 and is what the token limits below cite; its prices were not compared with these. This tier serves a different artefact from the open checkpoint — the model card calls it "the official version based on Qwen3.8-Flash-Next with more production features, e.g., 1M context length by default" — which is why the context here is 1M against the checkpoint's 262,144. Deliberately unscored: Artificial Analysis benchmarked the open weights, and its page for the served model returned 404 on 2026-09-25, so carrying the checkpoint's 40 across would assert an equivalence the publisher's own wording does not support. TOKEN LIMITS, recorded 2026-09-13: maxInputTokens 991,808 and maxOutputTokens 131,072. Qwen Cloud's page rounds them to 991K and 131K; Alibaba Model Studio's page for qwen3.8-flash prints Max Input Length 991808, Max Output Length 131072 and Context Window 1000000. The same table gives thinking mode a lower Max Input Length of 983616, which a scenario cannot select and is not stored, so a thinking-mode request with more than 983,616 input tokens, up to 991,808, passes here although Alibaba refuses it. Whether reasoning counts toward the output length is not stated and not recorded; the table's Max Chain-of-Thought Length of 262144 is larger than the output length, which suggests reasoning is counted apart from it. RE-READ 2026-09-25: the comparison this note said was not made has been made. Alibaba's Model Studio page for qwen3.8-flash prints, for the Singapore region, Input 0.15, Output 0.47, Input(Implicit Cache) 0.016, Explicit Cache Creation 0.2 and Explicit Cache Read 0.016 — the same five figures as Qwen Cloud's page — and its consolidated pricing table now carries a qwen3.8-flash row at $0.15 and $0.47 up to 1M input tokens. With two of Alibaba's own sources agreeing on every figure, confidence is high. The capability table marks Batch Inference unsupported in every region, so there is no batch tier to add. The page also prices a Global deployment scope, offered from Hong Kong, Frankfurt, Tokyo and Virginia, at $0.113 input, $0.382 output, $0.014 for either cache read and $0.177 for creation; this entry keeps the International scope, as every Alibaba tier here does.
Priced identically to Claude Fable 5 on every category, which is why the two rows read the same: $10 base, $12.50 five-minute cache write, $20 one-hour cache write, $1 cache hit, $50 output, and $5/$25 on the Batch API, which claude-mythos-5-batch now carries. Marked preview rather than current because the pricing table labels it “limited availability” and links it to anthropic.com/glasswing rather than to a model page. [CORRECTED 2026-08-29: this note said Anthropic documents the 1M window at flat pricing “for it by name”, and excused the missing batch row as the practice “elsewhere in this catalog”. Neither held. The pricing page's long-context sentence names “Claude 4.6 and later models and Claude Mythos Preview”: Mythos 5 is covered by the version class, not by name, and Mythos Preview is a different model, named separately because it carries no version number. The document that does name it is the context-windows page, now cited. And batch was encoded in ten of this catalog's rows at the time, including for the identically-priced Fable 5.] Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128k for Claude Mythos 5. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share. [RETIRED 2026-09-25, superseded by Claude Mythos 5.1. Anthropic's deprecations page still lists claude-mythos-5 as “Active”, with retirement “Not sooner than June 9, 2027”, and there is no legacy label to quote, because the overview's legacy list covers generally available models only. What Anthropic publishes instead is a migration: its Fable 5.1 migration guide has a section headed “Migrating to Claude Mythos 5.1 from Claude Mythos 5” and tells readers to update the model name “(or `claude-mythos-5` to `claude-mythos-5-1`)”, and the Mythos 5.1 model page calls that model “the same model as Claude Fable 5.1”, which has replaced Fable 5, the generally available model priced identically to this one. Mythos 5.1 is priced as this row except the cache read, $0.25 against $1.00, and is reached through the same Project Glasswing invitation, so a buyer with that access would choose it today, which is the lifecycle definition's test.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Published, and not carried until now: the batch table prints “Claude Mythos 5 (limited availability) | $5 / MTok | $25 / MTok”, an exact 50% of the standard row, while the identically-priced Fable 5 already had a batch row here. A reader comparing deferrable work against this model was being shown twice what Anthropic charges for it. Preview for the same reason as the standard row. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and no higher cap on the Batch API. Anthropic's thinking page prints 128k for Claude Mythos 5 and a dash in its batches beta ceiling column, and its batch-processing page does not name Claude Mythos 5 among the models the output-300k-2026-03-24 header raises. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. [RETIRED 2026-09-25, superseded by Claude Mythos 5.1. Anthropic's deprecations page still lists claude-mythos-5 as “Active”, with retirement “Not sooner than June 9, 2027”, and there is no legacy label to quote, because the overview's legacy list covers generally available models only. What Anthropic publishes instead is a migration: its Fable 5.1 migration guide has a section headed “Migrating to Claude Mythos 5.1 from Claude Mythos 5” and tells readers to update the model name “(or `claude-mythos-5` to `claude-mythos-5-1`)”, and the Mythos 5.1 model page calls that model “the same model as Claude Fable 5.1”, which has replaced Fable 5, the generally available model priced identically to this one. Mythos 5.1 is priced as this row except the cache read, $0.25 against $1.00, and is reached through the same Project Glasswing invitation, so a buyer with that access would choose it today, which is the lifecycle definition's test.] [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. No capability score is carried either, for the reason qwen3-8-flash-api states about its own: fast mode is a different serving configuration, Artificial Analysis measured the standard one, and copying the index across would assert an equivalence nobody has published. Fast mode is a research preview that buys faster output at double the standard rate: $10 input and $50 output against Opus 5's $5 and $25, for the same model. The cache figures here are not published separately — Anthropic states that prompt-caching multipliers apply on top of fast-mode pricing, so the five-minute write is 1.25x and a hit is 0.1x of the fast base, giving $12.50 and $1. The premium applies across the full context window including requests over 200k tokens, so there is no long-context surcharge to encode. Three things this row cannot express: fast mode is unavailable on the Batch API, it is first-party Claude API only rather than AWS or partner clouds, and the data-residency multiplier stacks on top of it as well. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and read from the model rather than from the mode. Anthropic's thinking page prints 128k as Claude Opus 5's `max_tokens` ceiling. The fast-mode page publishes no ceiling of its own: fast mode is a `speed: "fast"` request to claude-opus-5 and “runs the same model with a faster inference configuration”, so the model's ceiling is the one recorded, and it “is not available with the Batch API”, so no batch ceiling applies. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens, computed the way the five-minute write is: the one-hour multiplier, 2x, of the fast-mode base input, because the caching multipliers apply on top of fast-mode pricing. Which of the two a write pays is the scenario's one-hour share. [RETIRED 2026-09-25, superseded by fast mode on Claude Opus 5.5, for the reason Claude Opus 5 is superseded by Claude Opus 5.5: Anthropic's models overview lists Opus 5 under “Legacy models (still available)” and tells a reader to “start with Claude Opus 5.5 for most workloads”. Fast mode on Opus 5.5 costs $8 / $40 against this row's $10 / $50, and the same caching multipliers apply on top of it, so its computed cache rates are lower too. Anthropic's fast-mode page still lists Claude Opus 5 among the supported models; the lifecycle definition asks what a buyer would choose today, not what is still offered.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
The second of OpenAI's two Daybreak models, aliased as gpt-daybreak-red-latest; gpt-5.6-sol is the other, and this catalog already carries it. $12.50 input against Sol's $4.00 is 3.125x, and $75.00 against $20.00 is 3.75x. The cache-write figure is $15.625 as published, an exact 1.25x of base, and is carried unrounded because that is how the page prints it. [CORRECTED 2026-08-29: this row carried no context window and argued that none could be known, because the pricing table prints four dashes where Sol has a long-context row. But the pricing table states no window for Sol either — OpenAI puts windows on the model pages, which is where Sol's own 1,050,000 came from, and gpt-5.6-cyber has one: 400,000, with 128,000 max output tokens. Reading the silence of one page as evidence of absence left the catalog's most expensive tier looking unbounded beside siblings recorded at 1,050,000.] The two pages do disagree about one thing, and this row follows the pricing table. The model page carries the family's boilerplate line that prompts over 272K input tokens bill at 2x input and 1.5x output — the same sentence appears on gpt-5.4, gpt-5.5 and gpt-5.6-sol, all of which have 1,050,000-token windows and priced long-context columns. Cyber's own long-context columns are four dashes, so no long-context multiplier is encoded here; inventing $25.00 and $112.50 from a line that is not about this model would be a fabricated price. Batch is unavailable for Cyber: the batch table has no row for it. OpenAI warns that the gpt-daybreak-red-latest alias will be repointed at later frontier models with pricing adjusted to match, so this row describes gpt-5.6-cyber and not the alias. TOKEN LIMITS, recorded 2026-09-13: a maximum input of 272,000 tokens and an output cap of 128,000 inside the 400,000 window, from the model details on OpenAI's gpt-5.6-cyber page: “Maximum input tokens: 272,000” and “128,000 max output tokens”. That maximum input also means no prompt on this model can carry the “>272K input tokens” the boilerplate long-context line is about. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.”
Derived from an absence, and graded accordingly. The support table on OpenAI data residency page enumerates gpt-6-astra, gpt-5.6-sol, gpt-5.6-terra and gpt-5.6-luna and does not list gpt-5.6-cyber, while listing every one of its siblings. An argument from silence, so it is estimated rather than official-claim: the honest reading is that the endpoint does not serve this model, and the alternative is that the table is incomplete. Recorded as unavailable rather than left absent, because absent means nobody has checked and somebody has. Read 2026-09-10.
Google's cheapest current tier and, until now, absent from this catalog altogether, which left every comparison here priced against tiers a reader weighing high-volume work would not choose. $0.30 in, $2.50 out, $0.03 cached, one price for text, image, video and audio alike. Marked stable on Google's model list. No cache-write fee exists to encode: Google charges for cache reads and for storage, not for writes. Google also bills held prefixes at $1.00 per 1M tokens per hour, read 2026-09-25, and the cheaper the per-token rate the more that fee matters relative to it. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.5-flash-lite. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
Prices stored rather than a multiplier, because the discount is not uniform and a single scalar would misprice a bucket. Input and output do halve, $0.30 to $0.15 and $2.50 to $1.25, but context caching goes $0.03 to $0.02, which is two thirds and not a half. Batch and Flex are published at the same figures. Cache storage stays at $1.00 per 1M tokens per hour and is not discounted at all. This is the same shape as gemini-2-5-pro-batch, where the cached rate does not move either. CONFLICT, recorded 2026-09-26: Google's Batch API page, last updated 2026-09-17 and read 2026-09-25, says that a batch request which hits a cache pays “the standard context caching rates”. The pricing page, updated 2026-09-24 and the page the Batch API page sends readers to for details, prints this row's cache price at $0.02, against $0.03 on the Standard row, and the pricing page's row is what is carried. If the Batch API sentence means the Standard row, a cache hit in a batch costs 1.5 times what the calculator charges here. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.5-flash-lite. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
The other Flash-Lite Google sells, and cheaper per input token than the 3.5 while charging more for it on audio: $0.25 for text, image and video against $0.50 for audio, and $0.025 against $0.05 cached. This calculator prices one token stream, so the text figure is what is stored and the audio premium is not modelled. $1.50 output. Marked stable on Google's model list. Google also bills held prefixes at $1.00 per 1M tokens per hour, read 2026-09-25, and the cheaper the per-token rate the more that fee matters relative to it. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. No capability score: Artificial Analysis's page for the stable model returned 404 on 2026-09-25. Its page for the preview build, gemini-3-1-flash-lite-preview, prints 16 on Intelligence Index v4.3.2 and marks that build deprecated in favour of Gemini 3.5 Flash-Lite; a preview's score is not carried to a stable release nobody has published a measurement of. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.1-flash-lite. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
Here a single multiplier is exactly right, unlike its 3.5 sibling: $0.125 input, $0.75 output and $0.0125 cached are each precisely half the standard figures, and Batch and Flex are published at the same rates. Cache storage halves too, $1.00 to $0.50 per 1M tokens per hour: the one batch row Google printed on 2026-09-25 that discounts it, recorded since 2026-09-26 as a batchCacheStorageMultiplier of 0.5. CONFLICT, recorded 2026-09-26: Google's Batch API page, last updated 2026-09-17 and read 2026-09-25, says that a batch request which hits a cache pays “the standard context caching rates”. The pricing page, updated 2026-09-24 and the page the Batch API page sends readers to for details, prints this row's cache price at $0.0125, against $0.025 on the Standard row, and the pricing page's row is what is carried. If the Batch API sentence means the Standard row, a cache hit in a batch costs twice what the calculator charges here. TOKEN LIMITS, recorded 2026-09-13: an input token limit of 1,048,576 and an output token limit of 65,536, from Google's API model reference for gemini-3.1-flash-lite. Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.”
Same input, output and cache-write rates as Claude Fable 5, and one change: a cache read costs $0.25/M against Fable 5's $1.00/M. Anthropic states it as a 0.025x multiplier on base input where every other Claude model uses 0.1x, and frames the effect as roughly 25% cheaper for typical workloads and up to 45% for highly agentic ones — entirely from that one field. [CORRECTED 2026-09-25: this note said Fable 5 was not superseded, because Anthropic's pricing table listed both with no deprecation tag. That tested whether Fable 5 was still sold, not whether a buyer would choose it, which is what the lifecycle definition asks, and Fable 5 is now retired in favour of this tier; its own note gives the reasons. The same reading settles the release date, which this note gave as press reporting because Anthropic's announcement says only "September 2026": Anthropic's model page for Fable 5.1 prints “Released September 1, 2026”.] Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128k for Claude Fable 5.1. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share.
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Official Batch API 50% multiplier. Same input, output and cache-write rates as Claude Fable 5, and one change: a cache read costs $0.25/M against Fable 5's $1.00/M. Anthropic states it as a 0.025x multiplier on base input where every other Claude model uses 0.1x, and frames the effect as roughly 25% cheaper for typical workloads and up to 45% for highly agentic ones — entirely from that one field. [CORRECTED 2026-09-25: this note said Fable 5 was not superseded, because Anthropic's pricing table listed both with no deprecation tag. That tested whether Fable 5 was still sold, not whether a buyer would choose it, which is what the lifecycle definition asks, and Fable 5 is now retired in favour of this tier; its own note gives the reasons. The same reading settles the release date, which this note gave as press reporting because Anthropic's announcement says only "September 2026": Anthropic's model page for Fable 5.1 prints “Released September 1, 2026”.] Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-13: 128,000 tokens, reasoning included, and no higher cap on the Batch API. Anthropic's thinking page prints 128k for Claude Fable 5.1 and a dash in its batches beta ceiling column, and its batch-processing page does not name Claude Fable 5.1 among the models the output-300k-2026-03-24 header raises. Anthropic's k is 1,000 in these limits: the same table prints as 300k the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-10.
Read from OpenAI's own embedded pricing row, ["gpt-6-astra",10,1,12.5,50] for standard and ["gpt-6-astra",5,0.5,6.25,25] for batch — exactly half on all four columns. Above 272K input tokens the published long-context row is $20/$2/$25/$75, which is 2x on input, cached input and cache write but only 1.5x on output; the two multipliers are recorded separately for that reason. The 2026-09-03 release date is press reporting: openai.com's own announcement returned HTTP 403 to an automated fetch on both attempts, and the docs pages carry no date. TOKEN LIMITS, recorded 2026-09-13: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-6-astra page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.”
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists this model as eligible, and every OpenAI tier in this catalog is priced from 2026-07-30 or later, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-10.
Published as its own row at exactly half the standard rate on all four columns rather than as a stated multiplier. Read from OpenAI's own embedded pricing row, ["gpt-6-astra",10,1,12.5,50] for standard and ["gpt-6-astra",5,0.5,6.25,25] for batch — exactly half on all four columns. Above 272K input tokens the published long-context row is $20/$2/$25/$75, which is 2x on input, cached input and cache write but only 1.5x on output; the two multipliers are recorded separately for that reason. The 2026-09-03 release date is press reporting: openai.com's own announcement returned HTTP 403 to an automated fetch on both attempts, and the docs pages carry no date. TOKEN LIMITS, recorded 2026-09-13: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-6-astra page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.” RE-READ 2026-09-25: OpenAI's Flex table carries the identical row, ["gpt-6-astra", 5, 0.5, 6.25, 25] with $10 / $1 / $12.50 / $37.50 above 272K, and the model page says “Batch and Flex are priced at 50% of Standard rates”, so this tier stands for both arrangements, as the GPT-5.6 Flex tiers do, and its label now says so.
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists this model as eligible, and every OpenAI tier in this catalog is priced from 2026-07-30 or later, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-10. Applied multiplicatively on top of this arrangement discount, which is a derivation: OpenAI states the uplift and the discount separately and never their interaction. Anthropic documents multiplicative composition for the equivalent case, which is why it is the reading taken, and is not a statement about OpenAI.
Priced identically to Gemini 3.7 Flash on every published figure, promotional through 2026-12-31 and doubling on 2027-01-01. Google bills no cache write and instead bills cache storage by the hour: $0.50 per 1M tokens per hour through 2026-12-31 and $1.00 from 2027-01-01, read 2026-09-25. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. CONTEXT WINDOW: 1,048,576, the input token limit on Google's API model reference for gemini-3.8-flash, which gives an output token limit of 65,536 beside it. CHANGED 2026-09-13 from 1,000,000, which this note called “a gap in the evidence rather than a smaller window” because “every Google page for 3.8 Flash states only "1M"”. Read on 2026-09-13, the model reference prints the digits; whether it did when that was written is not known. [CORRECTED 2026-09-13: the note also pointed to “the DeepMind model card that gives 3.7's precise figure”. That card gives no precise figure. It says “a token context window of up to 1M”, as gemini-3-7-flash-card has recorded since 2026-08-29. The DeepMind Flash page for 3.8 lists Input tokens 1M and Output tokens 64k, the same two limits rounded.] Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. TOKEN LIMITS, recorded 2026-09-13: maxInputTokens and maxOutputTokens hold the two limits above. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.” Gemini 3.7 Flash is NOT superseded by this: Google's own announcement says 3.7 "remains fully supported for efficiency-first workloads", its model list tags both as stable side by side, and its deprecation schedule gives 3.7 no shutdown date.
Batch and Flex are published at exactly half the standard rate on input, output and context caching alike, so one multiplier is the right shape. Priced identically to Gemini 3.7 Flash on every published figure, promotional through 2026-12-31 and doubling on 2027-01-01. Google bills no cache write and instead bills cache storage by the hour: $0.50 per 1M tokens per hour through 2026-12-31 and $1.00 from 2027-01-01, read 2026-09-25, and this batch row does not discount it. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. CONTEXT WINDOW: 1,048,576, the input token limit on Google's API model reference for gemini-3.8-flash, which gives an output token limit of 65,536 beside it. CHANGED 2026-09-13 from 1,000,000, which this note called “a gap in the evidence rather than a smaller window” because “every Google page for 3.8 Flash states only "1M"”. Read on 2026-09-13, the model reference prints the digits; whether it did when that was written is not known. [CORRECTED 2026-09-13: the note also pointed to “the DeepMind model card that gives 3.7's precise figure”. That card gives no precise figure. It says “a token context window of up to 1M”, as gemini-3-7-flash-card has recorded since 2026-08-29. The DeepMind Flash page for 3.8 lists Input tokens 1M and Output tokens 64k, the same two limits rounded.] Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. CONFLICT, recorded 2026-09-26: Google's Batch API page, last updated 2026-09-17 and read 2026-09-25, says that a batch request which hits a cache pays “the standard context caching rates”. The pricing page, updated 2026-09-24 and the page the Batch API page sends readers to for details, prints this row's cache price at $0.0375 through 2026-12-31, against $0.075 on the Standard row, and the pricing page's row is what is carried. If the Batch API sentence means the Standard row, a cache hit in a batch costs twice what the calculator charges here. TOKEN LIMITS, recorded 2026-09-13: maxInputTokens and maxOutputTokens hold the two limits above. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.” Gemini 3.7 Flash is NOT superseded by this: Google's own announcement says 3.7 "remains fully supported for efficiency-first workloads", its model list tags both as stable side by side, and its deprecation schedule gives 3.7 no shutdown date.
The rate Google publishes as taking effect 2027-01-01, exactly double the promotional rate on every category, so a horizon that outlives the promotion can be priced against what will actually be charged. Priced identically to Gemini 3.7 Flash on every published figure, promotional through 2026-12-31 and doubling on 2027-01-01. Google bills no cache write and instead bills cache storage by the hour: $1.00 per 1M tokens per hour from 2027-01-01, the period this row covers, read 2026-09-25. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. CONTEXT WINDOW: 1,048,576, the input token limit on Google's API model reference for gemini-3.8-flash, which gives an output token limit of 65,536 beside it. CHANGED 2026-09-13 from 1,000,000, which this note called “a gap in the evidence rather than a smaller window” because “every Google page for 3.8 Flash states only "1M"”. Read on 2026-09-13, the model reference prints the digits; whether it did when that was written is not known. [CORRECTED 2026-09-13: the note also pointed to “the DeepMind model card that gives 3.7's precise figure”. That card gives no precise figure. It says “a token context window of up to 1M”, as gemini-3-7-flash-card has recorded since 2026-08-29. The DeepMind Flash page for 3.8 lists Input tokens 1M and Output tokens 64k, the same two limits rounded.] Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. TOKEN LIMITS, recorded 2026-09-13: maxInputTokens and maxOutputTokens hold the two limits above. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.” Gemini 3.7 Flash is NOT superseded by this: Google's own announcement says 3.7 "remains fully supported for efficiency-first workloads", its model list tags both as stable side by side, and its deprecation schedule gives 3.7 no shutdown date.
Half the list rate on every category, from the same published table. Priced identically to Gemini 3.7 Flash on every published figure, promotional through 2026-12-31 and doubling on 2027-01-01. Google bills no cache write and instead bills cache storage by the hour: $1.00 per 1M tokens per hour from 2027-01-01, read 2026-09-25, and this batch row does not discount it. Since 2026-09-26 the calculator charges it for the minutes Cache held per request sets; at the default of 0 minutes it charges none, and a caution says what each minute would add. CONTEXT WINDOW: 1,048,576, the input token limit on Google's API model reference for gemini-3.8-flash, which gives an output token limit of 65,536 beside it. CHANGED 2026-09-13 from 1,000,000, which this note called “a gap in the evidence rather than a smaller window” because “every Google page for 3.8 Flash states only "1M"”. Read on 2026-09-13, the model reference prints the digits; whether it did when that was written is not known. [CORRECTED 2026-09-13: the note also pointed to “the DeepMind model card that gives 3.7's precise figure”. That card gives no precise figure. It says “a token context window of up to 1M”, as gemini-3-7-flash-card has recorded since 2026-08-29. The DeepMind Flash page for 3.8 lists Input tokens 1M and Output tokens 64k, the same two limits rounded.] Google prints no combined window beside those two: its token guide defines the context window as “the combined limit of input and output tokens” and says to find the size on the models page, which gives these two limits. contextTokens holds the input limit, the smallest window that limit allows, so a request needing more than 1,048,576 tokens in total is refused here although Google may accept it. CONFLICT, recorded 2026-09-26: Google's Batch API page, last updated 2026-09-17 and read 2026-09-25, says that a batch request which hits a cache pays “the standard context caching rates”. The pricing page, updated 2026-09-24 and the page the Batch API page sends readers to for details, prints this row's cache price at $0.075 from 2027-01-01, against $0.15 on the Standard row, and the pricing page's row is what is carried. If the Batch API sentence means the Standard row, a cache hit in a batch costs twice what the calculator charges here. TOKEN LIMITS, recorded 2026-09-13: maxInputTokens and maxOutputTokens hold the two limits above. Whether thinking counts toward the output limit is not recorded: the one statement found, in Google's thinking guide, is about the Interactions API's `max_output_tokens` parameter, which it says sets “the maximum number of tokens a response can generate, including thought tokens.” Gemini 3.7 Flash is NOT superseded by this: Google's own announcement says 3.7 "remains fully supported for efficiency-first workloads", its model list tags both as stable side by side, and its deprecation schedule gives 3.7 no shutdown date.
Meta's pricing page states outright that both versions share the same standard pricing, and the Contributor tier carries over unchanged. Muse Spark 1.2 is not marked superseded: Meta calls 1.3 "the latest version ... Recommended for new work" without retiring 1.2, and prices them identically. The 2026-09-02 release date is secondary press; neither Meta docs page carries one.
An opt-in tier at roughly a twentieth of the standard rate, priced on the condition that Meta trains future models on the requests. Meta's pricing page states outright that both versions share the same standard pricing, and the Contributor tier carries over unchanged. Muse Spark 1.2 is not marked superseded: Meta calls 1.3 "the latest version ... Recommended for new work" without retiring 1.2, and prices them identically. The 2026-09-02 release date is secondary press; neither Meta docs page carries one.
Read from the pricing page on 2026-09-11 under the model column deepseek-flash, version DeepSeek-V4.1-Flash. Listed figures are the PEAK rates, the convention every DeepSeek tier here uses: cache-miss input $0.30, cache hit $0.006, output $1.20, and off-peak rates are half of the peak rates, with peak hours 01:00-04:00 and 06:00-10:00 UTC Monday through Friday and all other hours off-peak. Effective 04:00 UTC on 2026-09-10 per the release note. NO SEPARATE CACHE-WRITE FEE: the Context Caching guide describes caching as enabled by default for all users and documents no write charge, so a miss is billed at the standard input rate and persisted as a side effect — encoded here as cacheWriteUsdPerM equal to input, the same as every other DeepSeek tier. NO LONG-CONTEXT TIER: the rate is flat to the full 1M window, so there is no threshold and no inclusive-or-exclusive question. THIS IS CHEAPER THAN THE FLASH TIER IT REPLACES, which is unusual enough to state: against deepseek-v4-flash-api-tod it is 0.68x on cache-miss input, 0.91x on output and 0.43x on cache hits, at every hour. [CORRECTED 2026-09-11: this sentence read "The concurrency limit rises from 500 to 2,500", which compared two different tiers and reported the comparison as a change over time. The pricing page publishes exactly two concurrency limits, one per model column: 2,500 against deepseek-flash, which is this tier, and 500 against deepseek-v4-pro. It carries no column for the retired V4-Flash tier at all, and neither the 2026-08-13 nor the 2026-09-10 release note states a limit for it, so there is no published figure for this tier to have risen from. What is publishable is the pair as it stands: 2,500 here, 500 on deepseek-v4-pro. The 2026-09-10 release note routes deepseek-v4-pro requests to this model at this price from 04:00 UTC on 2026-09-14 and says nothing about which limit they are then served under; the pricing page, read on 2026-09-13, says V4 Pro service continues after that date with its billing unchanged. deepseek-v4-pro-api-tod records the conflict.] OUTPUT CAP, recorded 2026-09-13: 393,216 tokens. The pricing page prints MAX OUTPUT as “MAXIMUM: 384K” in a row shared by both model columns, and DeepSeek's chat completion reference spells it out: max_tokens “must be between 1 and 384K (393216)”. That reference does not say whether thinking-mode reasoning counts toward max_tokens, so that is not recorded, and the pricing page publishes no input limit below the window. HOLIDAYS, recorded 2026-09-25: the pricing page's footnote now reads "Off-peak rates are half of the peak rates. Peak hours are 01:00 - 04:00 and 06:00 - 10:00 UTC, Monday through Friday, excluding Chinese public holidays. All other hours are off-peak, including weekends and Chinese public holidays in full." The same sentence as deepseek-v4-flash-api-tod quotes it, read on 2026-08-27, named no holidays. The schedule this tier carries cannot express a holiday calendar, so a weekday inside a Chinese public holiday is costed at its peak hours like any other weekday. That overstates the peak share, and so the cost, by as many such weekdays as a horizon holds; DeepSeek names no calendar, and none is modelled here. The conflict over deepseek-v4-pro that the 2026-09-13 reading recorded is resolved on deepseek-v4-pro-api-tod: V4 Pro is still served at its own rates, so no V4 Pro request is billed at this tier.
Released 2026-09-22, the date Anthropic's model page prints, at $4 input and $20 output against Opus 5's $5 and $25. The cache read moves most: $0.20/M, which Anthropic states as 0.05x the base input price where Opus 5 uses 0.1x, so Opus 5's $0.50 becomes $0.20. The five-minute cache write stays at 1.25x input, $5.00, and the one-hour one is $8.00. Anthropic's announcement says Opus 5.5 “costs 40% less to run than Opus 5”, which combines these rates with fewer tokens per task; the second half is Anthropic's claim about its own model and nothing here models it. The full 1M window is billed at standard rates, so no long-context surcharge applies. Retirement not sooner than 2027-09-22, per the same page. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-25: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128K for Claude Opus 5.5. Anthropic's k is 1,000 in these limits: the same table prints as 300K the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $8 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share.
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-25.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Official Batch API 50% multiplier: Anthropic's batch table prints $2 input and $10 output for Claude Opus 5.5, half the standard row. Released 2026-09-22, the date Anthropic's model page prints, at $4 input and $20 output against Opus 5's $5 and $25. The cache read moves most: $0.20/M, which Anthropic states as 0.05x the base input price where Opus 5 uses 0.1x, so Opus 5's $0.50 becomes $0.20. The five-minute cache write stays at 1.25x input, $5.00, and the one-hour one is $8.00. Anthropic's announcement says Opus 5.5 “costs 40% less to run than Opus 5”, which combines these rates with fewer tokens per task; the second half is Anthropic's claim about its own model and nothing here models it. The full 1M window is billed at standard rates, so no long-context surcharge applies. Retirement not sooner than 2027-09-22, per the same page. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-25: 128,000 tokens, reasoning included, and 300,000 with the output-300k-2026-03-24 beta header. Anthropic's thinking page prints 128K for Claude Opus 5.5 and a batches beta ceiling of 300K, and its batch-processing page names Claude Opus 5.5 among the models for whose batch requests that header “raises the `max_tokens` cap to 300,000”, on the Message Batches API only. An output between the two caps is in doubt rather than refused, because a scenario does not record whether a request sends the header. Anthropic's k is 1,000 in these limits: the same table prints as 300K the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward both caps. ONE-HOUR WRITE, recorded 2026-09-25: $8 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-25.
Graded estimated rather than official-claim, for the reason claude-opus-5-fast gives: two of the five figures here are read off Anthropic's table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. No capability score is carried either, for the reason qwen3-8-flash-api states about its own: fast mode is a different serving configuration, Artificial Analysis measured the standard one, and copying the index across would assert an equivalence nobody has published. Fast mode is a research preview that buys faster output at double the standard rate: $8 input and $40 output against Opus 5.5's $4 and $20, for the same model, and Anthropic's announcement gives the gain as “up to 2.5x speed”. The cache figures here are not published separately — Anthropic states that prompt-caching multipliers apply on top of fast-mode pricing, so the five-minute write is 1.25x and a hit is 0.05x of the fast base, giving $10 and $0.40. The premium applies across the full context window including requests over 200k tokens, so there is no long-context surcharge to encode. Three things this row cannot express: fast mode is unavailable on the Batch API, it is sold on the first-party Claude API only rather than on AWS or partner clouds, and access is by request, through an account manager or a waitlist. The data-residency multiplier stacks on top of it, which the record below carries. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-25: 128,000 tokens, reasoning included, and read from the model rather than from the mode. Anthropic's thinking page prints 128K as Claude Opus 5.5's `max_tokens` ceiling. The fast-mode page publishes no ceiling of its own: fast mode is a `speed: "fast"` request to claude-opus-5-5 and “runs the same model with a faster inference configuration”, so the model's ceiling is the one recorded, and it “is not available with the Batch API”, so no batch ceiling applies. Anthropic's k is 1,000 in these limits: the same table prints as 300K the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $16 per million tokens, computed the way the five-minute write is: the one-hour multiplier, 2x, of the fast-mode base input, because the caching multipliers apply on top of fast-mode pricing. Which of the two a write pays is the scenario's one-hour share.
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-25.
Claude Mythos 5.1 is, in the words of Anthropic's model page, “the same model as Claude Fable 5.1, offered by invitation only through Project Glasswing”, and it is priced identically to Fable 5.1 on every category: $10 base, $12.50 five-minute cache write, $20 one-hour cache write, $0.25 cache hit and $50 output. The cache hit is 0.025x the base input price where Claude Mythos 5 charges 0.1x, $1.00. Marked preview rather than current for the reason Mythos 5 is: the pricing table labels it “limited availability”, and the model page opens “Invite only.” Released 2026-09-01 and not retired sooner than 2027-09-01, per that page. No capability score: Artificial Analysis's page for this model returned 404 on 2026-09-25, and Fable 5.1's score is not carried across although Anthropic calls them the same model, because Anthropic's own Terminal-Bench 4.0 figures differ between them, 55.8% for Fable 5.1 and 60.9% for Mythos 5.1, which it puts down to the tasks its earlier safeguards intervened on. The Batch API halves input and output, which claude-mythos-5-1-batch carries. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-25: 128,000 tokens, reasoning included. Anthropic's thinking page lists each model's `max_tokens` ceiling and prints 128K for Claude Mythos 5.1, and its context-windows page names Claude Mythos 5.1 among the models with a 1M-token window. Anthropic's k is 1,000 in these limits: the same table prints as 300K the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens, read from the "1h cache writes" column of Anthropic's pricing table, which prices an hour-long entry at 2x base input against 1.25x for five minutes. Which of the two a write pays is the scenario's one-hour share.
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-25.
Graded estimated rather than official-claim: two of the five figures here are read off Anthropic's batch table and three are computed by this catalog, and an entity carries one grade, so it takes the weaker. That table prints batch input and batch output only; every cache price here is the standard one halved, because Anthropic's caching multipliers “stack with other pricing modifiers, including the Batch API discount”. Official Batch API 50% multiplier: Anthropic's batch table prints $5 input and $25 output for Claude Mythos 5.1, half the standard row. Preview for the same reason as the standard row. Claude Mythos 5.1 is, in the words of Anthropic's model page, “the same model as Claude Fable 5.1, offered by invitation only through Project Glasswing”, and it is priced identically to Fable 5.1 on every category: $10 base, $12.50 five-minute cache write, $20 one-hour cache write, $0.25 cache hit and $50 output. The cache hit is 0.025x the base input price where Claude Mythos 5 charges 0.1x, $1.00. Marked preview rather than current for the reason Mythos 5 is: the pricing table labels it “limited availability”, and the model page opens “Invite only.” Released 2026-09-01 and not retired sooner than 2027-09-01, per that page. No capability score: Artificial Analysis's page for this model returned 404 on 2026-09-25, and Fable 5.1's score is not carried across although Anthropic calls them the same model, because Anthropic's own Terminal-Bench 4.0 figures differ between them, 55.8% for Fable 5.1 and 60.9% for Mythos 5.1, which it puts down to the tasks its earlier safeguards intervened on. Uses the new-generation tokenizer (~30% more tokens for the same text than Sonnet 4.6 and earlier); raw per-token price comparisons across that boundary undercount effective cost. OUTPUT CAP, recorded 2026-09-25: 128,000 tokens, reasoning included, and no higher cap on the Batch API. Anthropic's thinking page prints 128K for Claude Mythos 5.1 and a dash in its batches beta ceiling column, and its batch-processing page does not name Claude Mythos 5.1 among the models the output-300k-2026-03-24 header raises. Anthropic's k is 1,000 in these limits: the same table prints as 300K the batch ceiling that the batch-processing page gives as 300,000. Thinking tokens “count toward `max_tokens` alongside the response text”, so reasoning counts toward the cap. ONE-HOUR WRITE, recorded 2026-09-25: $20 per million tokens listed, read from the "1h cache writes" column of Anthropic's pricing table, and discounted like the five-minute write, because Anthropic's caching multipliers "stack with other pricing modifiers, including the Batch API discount". Which of the two a write pays is the scenario's one-hour share. [DOWNGRADED 2026-09-26: graded official-claim until this date, although Anthropic prints none of the cache prices this tier shows.]
Anthropic prices US-only inference at 1.1x "across all token pricing categories (input tokens, output tokens, cache writes, and cache reads)". The multiplier stacks with the others rather than replacing them: the caching section states it applies on top of the Batch API discount and of prompt-caching multipliers, and the fast-mode section says the same. Requested per request as inference_geo: "us"; global is the default and is what every rate in this entry is. Read 2026-09-25.
Released 2026-09-22: OpenAI's API changelog entry of that date reads “Released GPT-6 Sol (`gpt-6-sol`) and GPT-6 Luna (`gpt-6-luna`).” Standard rates read from OpenAI's pricing table and the gpt-6-sol model page, which agree: $2.00 input, $0.20 cached input, $2.50 cache write and $10.00 output per 1M tokens, exactly half of GPT-5.6 Sol's $4.00 / $0.40 / $5.00 / $20.00 on every column, with the same window and limits. The cache write is 1.25x the input rate and a cached read 10% of it, as the model page states. Above 272K input tokens the published long-context row is $4.00 / $0.40 / $5.00 / $15.00: 2x on input, cached input and cache write and 1.5x on output, and “for the full request”, as the model page puts it. Fast mode, which the model page prices at “2x the applicable rates”, is not carried, as for GPT-6 Astra. TOKEN LIMITS, recorded 2026-09-25: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-6-sol page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.”
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists gpt-6-sol as eligible, and the model was released on 2026-09-22, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-25.
Published as its own row at exactly half the standard rate on all four columns, in OpenAI's Batch table and in its Flex table alike: $1.00 / $0.10 / $1.25 / $5.00, and $2.00 / $0.20 / $2.50 / $7.50 above 272K. The model page says “Batch and Flex are priced at 50% of Standard rates”, so this tier stands for both arrangements, as GPT-6 Astra's does, and carries the standard rates below with a 0.5 multiplier. Released 2026-09-22: OpenAI's API changelog entry of that date reads “Released GPT-6 Sol (`gpt-6-sol`) and GPT-6 Luna (`gpt-6-luna`).” Standard rates read from OpenAI's pricing table and the gpt-6-sol model page, which agree: $2.00 input, $0.20 cached input, $2.50 cache write and $10.00 output per 1M tokens, exactly half of GPT-5.6 Sol's $4.00 / $0.40 / $5.00 / $20.00 on every column, with the same window and limits. The cache write is 1.25x the input rate and a cached read 10% of it, as the model page states. Above 272K input tokens the published long-context row is $4.00 / $0.40 / $5.00 / $15.00: 2x on input, cached input and cache write and 1.5x on output, and “for the full request”, as the model page puts it. Fast mode, which the model page prices at “2x the applicable rates”, is not carried, as for GPT-6 Astra. TOKEN LIMITS, recorded 2026-09-25: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-6-sol page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.” In-region routing is narrower here than on the standard tier; the residency record below says how.
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists gpt-6-sol as eligible, and the model was released on 2026-09-22, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-25. Applied multiplicatively on top of this arrangement discount, which is a derivation: OpenAI states the uplift and the discount separately and never their interaction. Anthropic documents multiplicative composition for the equivalent case, which is why it is the reading taken, and is not a statement about OpenAI. The region matters here as it does not on the standard tier: “EU data residency is available only with Standard processing”, and the support table notes “Europe (EEA + Switzerland): `gpt-6-sol` or `gpt-6-luna`: Standard processing only” beside the Batch endpoint, so at this tier's rates the United States endpoint is the one that pins the routing.
Released 2026-09-22: OpenAI's API changelog entry of that date reads “Released GPT-6 Sol (`gpt-6-sol`) and GPT-6 Luna (`gpt-6-luna`).” Standard rates read from OpenAI's pricing table and the gpt-6-luna model page, which agree: $0.10 input, $0.01 cached input, $0.125 cache write and $0.50 output per 1M tokens, half of GPT-5.6 Luna's $0.20 / $0.02 / $0.25 on input, cached input and cache write, and $0.50 against its $1.20 on output, with the same window and limits. The cache write is 1.25x the input rate and a cached read 10% of it, as the model page states. Above 272K input tokens the published long-context row is $0.20 / $0.02 / $0.25 / $0.75: 2x on input, cached input and cache write and 1.5x on output, and “for the full request”, as the model page puts it. Fast mode, which the model page prices at “2x the applicable rates”, is not carried, as for GPT-6 Astra. TOKEN LIMITS, recorded 2026-09-25: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-6-luna page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.”
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists gpt-6-luna as eligible, and the model was released on 2026-09-22, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-25.
Published as its own row at exactly half the standard rate on all four columns, in OpenAI's Batch table and in its Flex table alike: $0.05 / $0.005 / $0.0625 / $0.25, and $0.10 / $0.01 / $0.125 / $0.375 above 272K. The model page says “Batch and Flex are priced at 50% of Standard rates”, so this tier stands for both arrangements, as GPT-6 Astra's does, and carries the standard rates below with a 0.5 multiplier. Released 2026-09-22: OpenAI's API changelog entry of that date reads “Released GPT-6 Sol (`gpt-6-sol`) and GPT-6 Luna (`gpt-6-luna`).” Standard rates read from OpenAI's pricing table and the gpt-6-luna model page, which agree: $0.10 input, $0.01 cached input, $0.125 cache write and $0.50 output per 1M tokens, half of GPT-5.6 Luna's $0.20 / $0.02 / $0.25 on input, cached input and cache write, and $0.50 against its $1.20 on output, with the same window and limits. The cache write is 1.25x the input rate and a cached read 10% of it, as the model page states. Above 272K input tokens the published long-context row is $0.20 / $0.02 / $0.25 / $0.75: 2x on input, cached input and cache write and 1.5x on output, and “for the full request”, as the model page puts it. Fast mode, which the model page prices at “2x the applicable rates”, is not carried, as for GPT-6 Astra. TOKEN LIMITS, recorded 2026-09-25: a maximum input of 922,000 tokens and an output cap of 128,000 inside the 1,050,000 window, from the model details on OpenAI's gpt-6-luna page: “Maximum input tokens: 922,000” and “128,000 max output tokens”. Reasoning counts toward the output cap: OpenAI's token-counting guide says the `max_output_tokens` and `max_completion_tokens` parameters “limit all tokens generated by the model, including non-visible tokens.” In-region routing is narrower here than on the standard tier; the residency record below says how.
OpenAI states: "Data residency endpoints are charged a 10% uplift for models released on or after March 5, 2026, that are eligible for data residency." The support table on the same page lists gpt-6-luna as eligible, and the model was released on 2026-09-22, so both conditions hold. Two things OpenAI does not say, and this catalog does not invent: which token categories the uplift covers - Anthropic enumerates input, output, cache reads and cache writes, OpenAI writes only "10% uplift" against the endpoint - and how it composes with the Batch and Flex discounts. Read 2026-09-25. Applied multiplicatively on top of this arrangement discount, which is a derivation: OpenAI states the uplift and the discount separately and never their interaction. Anthropic documents multiplicative composition for the equivalent case, which is why it is the reading taken, and is not a statement about OpenAI. The region matters here as it does not on the standard tier: “EU data residency is available only with Standard processing”, and the support table notes “Europe (EEA + Switzerland): `gpt-6-sol` or `gpt-6-luna`: Standard processing only” beside the Batch endpoint, so at this tier's rates the United States endpoint is the one that pins the routing.
Released 2026-09-21, the date of xAI's release-notes entry and of its announcement, which says Grok 4.7 is “Served at the same price and speed as Grok 4.6” and “uses a new, larger base model compared to Grok 4.6”. The prices bear the first half out: $2.00 input, $0.50 cached input and $6.00 output per 1M tokens, doubling on all three for the whole request once the prompt reaches 200K tokens ($4.00 / $1.00 / $12.00), identical to Grok 4.6 on the pricing page and on the model page. The boundary is inclusive, as on Grok 4.6: “Models with long context pricing bill the long context rates for all tokens in a request once its prompt reaches the model's long context threshold.” Batch is unavailable, not merely undiscounted: the model page gives Batch API as “Not supported”, and the pricing page's batch discounts name only the Grok 4.3 and 4.20 lines. Grok 4.7 Fast, the same model at twice the token rates, is sold only through Cursor and Grok Build and not on the xAI API, so it is not carried. xAI prices a US regional endpoint at 1.1x for this model and Grok 4.6; no in-region record is carried for any xAI tier yet, so the calculator reports that price as unrecorded. OUTPUT LIMIT, read 2026-09-25: xAI's Grok 4.7 guide gives the output limit as “No text output limit” and the context window as “500,000 tokens”, so the window is the only limit recorded.
xAI states: “Token usage on the US endpoint costs 10% more than on the global endpoint. The premium applies to input, output, and cached input tokens, including long-context rates, and prompt caching discounts still apply.” The same page names this model as one of the two the US endpoint serves: “The US endpoint currently serves two models, grok-4.7 and grok-4.6.” xAI publishes no cache-write price, so the engine bills a write as input, and the premium covers input. Global is the default and is what every rate in this entry is. Read 2026-09-25.
Priced exactly as MiMo V2.5 Pro: Xiaomi's pay-as-you-go page lists mimo-v2.6-pro on the same overseas row as mimo-v2.5-pro, which it marks “(to be deprecated)”, at $0.0036 cache hit, $0.435 cache miss and $0.87 output per 1M tokens, and the V2.6 announcement says “The MiMo-V2.6 series adopts the same API pricing as the V2.5 series.” The write bucket is zero because the page prints “Cache Write: Limited-time Free”: a promotion with no published end date, which can be withdrawn without the model changing. No effectiveFrom is recorded: Xiaomi's announcement carries no date beyond an “Update Time September 22, 2026” footer, and Artificial Analysis gives the release as 2026-09-21. Xiaomi quotes Artificial Analysis's 46.32 on its model page. mimo-v2.6-pro-ultraspeed, priced at ten times this row ($4.35 cache miss, $8.70 output, $0.036 cache hit) and without batch, is not carried. CONTEXT, recorded 2026-09-25 and settled 2026-09-26: Xiaomi's model page prints “1M tokens”, which does not distinguish 1,000,000 from 2^20, and the weights' own config.json declares max_position_embeddings 1048576, so 1,048,576 stands. On 2026-09-25 that config could not be read, Hugging Face answering HTTP 401, and this entry was graded medium for it. Hugging Face gives the same answer for XiaomiMiMo/MiMo-V2.6-Pro, a name no repository has; the weights are XiaomiMiMo/MiMo-V2.6-Pro-RL. OUTPUT CAP, recorded 2026-09-25: 131,072 tokens, reasoning included. Xiaomi's model page lists a max output of 128K tokens, and its chat API reference gives max_completion_tokens a default of 131072 for mimo-v2.6-pro and a required range of [1, 131072], and describes it as “An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.” Batch is published at half of every rate, which mimo-v2-6-pro-api-batch carries.
Xiaomi's Batch API row prints $0.0018 cache hit, $0.2175 cache miss and $0.435 output for mimo-v2.6-pro, exactly half the real-time row on all three columns, so it is carried as that row with a 0.5 multiplier, as this catalog does wherever a discount is uniform. The same page says mimo-v2.5-pro and mimo-v2.5 do not support the Batch API, so this arrangement is new with V2.6. Priced exactly as MiMo V2.5 Pro: Xiaomi's pay-as-you-go page lists mimo-v2.6-pro on the same overseas row as mimo-v2.5-pro, which it marks “(to be deprecated)”, at $0.0036 cache hit, $0.435 cache miss and $0.87 output per 1M tokens, and the V2.6 announcement says “The MiMo-V2.6 series adopts the same API pricing as the V2.5 series.” The write bucket is zero because the page prints “Cache Write: Limited-time Free”: a promotion with no published end date, which can be withdrawn without the model changing. No effectiveFrom is recorded: Xiaomi's announcement carries no date beyond an “Update Time September 22, 2026” footer, and Artificial Analysis gives the release as 2026-09-21. Xiaomi quotes Artificial Analysis's 46.32 on its model page. mimo-v2.6-pro-ultraspeed, priced at ten times this row ($4.35 cache miss, $8.70 output, $0.036 cache hit) and without batch, is not carried. CONTEXT, recorded 2026-09-25 and settled 2026-09-26: Xiaomi's model page prints “1M tokens”, which does not distinguish 1,000,000 from 2^20, and the weights' own config.json declares max_position_embeddings 1048576, so 1,048,576 stands. On 2026-09-25 that config could not be read, Hugging Face answering HTTP 401, and this entry was graded medium for it. Hugging Face gives the same answer for XiaomiMiMo/MiMo-V2.6-Pro, a name no repository has; the weights are XiaomiMiMo/MiMo-V2.6-Pro-RL. OUTPUT CAP, recorded 2026-09-25: 131,072 tokens, reasoning included. Xiaomi's model page lists a max output of 128K tokens, and its chat API reference gives max_completion_tokens a default of 131072 for mimo-v2.6-pro and a required range of [1, 131072], and describes it as “An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.”
Priced exactly as MiMo V2.5: Xiaomi's pay-as-you-go page lists mimo-v2.6-flash on the same overseas row as mimo-v2.5, which it marks “(to be deprecated)”, at $0.0028 cache hit, $0.14 cache miss and $0.28 output per 1M tokens, and the V2.6 announcement says “The MiMo-V2.6 series adopts the same API pricing as the V2.5 series.” The write bucket is zero because the page prints “Cache Write: Limited-time Free”: a promotion with no published end date, which can be withdrawn without the model changing. No effectiveFrom is recorded: Xiaomi's announcement carries no date beyond an “Update Time September 22, 2026” footer, and Artificial Analysis gives the release as 2026-09-21. [CORRECTED 2026-10-07: this read “No capability score: Artificial Analysis's page for this model returned 404 on 2026-09-25.” The page answers now and prints 38 on Intelligence Index v4.3.2, not marked an estimate, so this tier carries the score its model carries.] Xiaomi's announcement says it “has comprehensively outperformed MiMo-V2.5-Pro”, which is Xiaomi's claim about its own model. CONTEXT, recorded 2026-09-25 and settled 2026-09-26: Xiaomi's model page prints “1M tokens”, which does not distinguish 1,000,000 from 2^20, and the weights' own config.json declares max_position_embeddings 1048576, so 1,048,576 stands. On 2026-09-25 that config could not be read, Hugging Face answering HTTP 401, and this entry was graded medium for it. Hugging Face gives the same answer for XiaomiMiMo/MiMo-V2.6-Flash, a name no repository has; the weights are XiaomiMiMo/MiMo-V2.6-Flash-RL. OUTPUT CAP, recorded 2026-09-25: 131,072 tokens, reasoning included. Xiaomi's model page lists a max output of 128K tokens, and its chat API reference gives max_completion_tokens a default of 131072 for mimo-v2.6-flash and a required range of [1, 131072], and describes it as “An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.” Batch is published at half of every rate, which mimo-v2-6-flash-api-batch carries.
Xiaomi's Batch API row prints $0.0014 cache hit, $0.07 cache miss and $0.14 output for mimo-v2.6-flash, exactly half the real-time row on all three columns, so it is carried as that row with a 0.5 multiplier, as this catalog does wherever a discount is uniform. The same page says mimo-v2.5-pro and mimo-v2.5 do not support the Batch API, so this arrangement is new with V2.6. Priced exactly as MiMo V2.5: Xiaomi's pay-as-you-go page lists mimo-v2.6-flash on the same overseas row as mimo-v2.5, which it marks “(to be deprecated)”, at $0.0028 cache hit, $0.14 cache miss and $0.28 output per 1M tokens, and the V2.6 announcement says “The MiMo-V2.6 series adopts the same API pricing as the V2.5 series.” The write bucket is zero because the page prints “Cache Write: Limited-time Free”: a promotion with no published end date, which can be withdrawn without the model changing. No effectiveFrom is recorded: Xiaomi's announcement carries no date beyond an “Update Time September 22, 2026” footer, and Artificial Analysis gives the release as 2026-09-21. [CORRECTED 2026-10-07: this read “No capability score: Artificial Analysis's page for this model returned 404 on 2026-09-25.” The page answers now and prints 38 on Intelligence Index v4.3.2, not marked an estimate, so this tier carries the score its model carries.] Xiaomi's announcement says it “has comprehensively outperformed MiMo-V2.5-Pro”, which is Xiaomi's claim about its own model. CONTEXT, recorded 2026-09-25 and settled 2026-09-26: Xiaomi's model page prints “1M tokens”, which does not distinguish 1,000,000 from 2^20, and the weights' own config.json declares max_position_embeddings 1048576, so 1,048,576 stands. On 2026-09-25 that config could not be read, Hugging Face answering HTTP 401, and this entry was graded medium for it. Hugging Face gives the same answer for XiaomiMiMo/MiMo-V2.6-Flash, a name no repository has; the weights are XiaomiMiMo/MiMo-V2.6-Flash-RL. OUTPUT CAP, recorded 2026-09-25: 131,072 tokens, reasoning included. Xiaomi's model page lists a max output of 128K tokens, and its chat API reference gives max_completion_tokens a default of 131072 for mimo-v2.6-flash and a required range of [1, 131072], and describes it as “An upper bound for the number of tokens that can be generated for a completion, including visible output tokens and reasoning tokens.”
The SGLang cookbook's B300 Low-Latency cell, marked Verified: TP8 on the native MXFP4 checkpoint with a bf16 cache and no speculation, 785 tok/s per GPU at concurrency 16 on requests of 8,192 prompt and 1,024 generated tokens. That figure counts both kinds of token, so the eight GPUs served 6,280 tok/s, 698 decode tok/s generated and 5,582 prefill tok/s of prompt. The cell's own medians close the loop: 16 streams at P50 TTFT 3,539 ms and P50 TPOT 19.47 ms generate 16 × 1,024 / (3.539 + 1,024 × 0.01947) = 697.9 tok/s. It is the low case because this recipe trades throughput for per-user speed, not because it reads the same run pessimistically. Decode and prefill split that one run's tokens at its own shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which reproduces the run only at 8,192 tokens in and 1,024 out. At another prompt share and the same request length it understates, because one server can move GPU time between the two phases and the calculator holds each limit fixed. For requests longer than 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the run. The run's random prompts shared no prefixes, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from this system than the calculator credits. [CORRECTED 2026-09-13: this profile was 12,000 decode and 120,000 prefill tok/s, SGLang's 2,633 to 2,808 tok/s per GPU read as decode throughput on 8 GB300 and derated by 0.57, with prefill set at ten times decode. Those figures count prompt tokens as well and divide by the 16 or 32 GPUs of prefill/decode-disaggregated arms.]
The SGLang cookbook's B300 Balanced cell, marked Verified: TP8 with decode context parallelism across the eight GPUs, on the native MXFP4 checkpoint with a bf16 cache and no speculation, 1,395 tok/s per GPU at concurrency 64 on requests of 8,192 prompt and 1,024 generated tokens. That figure counts both kinds of token, so the eight GPUs served 11,160 tok/s, 1,240 decode tok/s generated and 9,920 prefill tok/s of prompt. At that shape both limits give 1.211 requests/s, and the cell's own medians agree: 64 streams at P50 TTFT 11,635 ms and P50 TPOT 40.19 ms complete 64 / (11.635 + 1,024 × 0.04019) = 1.212 requests/s, 1,241.5 generated tok/s. The cookbook publishes no Balanced point past concurrency 64 because the KDA state pool clamps admission at 101 running requests. Decode and prefill split that one run's tokens at its own shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which reproduces the run only at 8,192 tokens in and 1,024 out. At another prompt share and the same request length it understates, because one server can move GPU time between the two phases and the calculator holds each limit fixed. For requests longer than 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the run. The run's random prompts shared no prefixes, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from this system than the calculator credits. [CORRECTED 2026-09-13: this profile was 21,000 decode and 210,000 prefill tok/s, SGLang's 2,633 tok/s per GPU read as decode throughput and multiplied by 8 GB300, with prefill set at ten times decode. That figure counts prompt tokens as well and divides by the 32 GPUs of a two-prefill, two-decode arm. The note also called the 3,000 tok/s planning guess carried before the weights existed low by roughly a factor of seven; that guess was 2.4 times this rate.]
The SGLang cookbook's B300 Balanced DSPARK cell: the Balanced recipe with DSPARK speculative decoding and --max-running-requests 256, 1,987 tok/s per GPU at concurrency 64 on requests of 8,192 prompt and 1,024 generated tokens. That figure counts both kinds of token, so the eight GPUs served 15,896 tok/s, 1,766 decode tok/s generated and 14,130 prefill tok/s of prompt, and the cell's medians agree: 64 × 1,024 / (12.038 + 1,024 × 0.02447) = 1,766.7 generated tok/s. Graded low because the cookbook pins the draft acceptance length with SGLANG_SIMULATE_ACC_LEN=4.5, so the cell reports what DSPARK's block of 7 delivers at that acceptance rather than an acceptance measured on this workload, and the cookbook asks readers to measure against the same recipe without speculation before adopting it. SGLang pull request #39029, which added the MI350X cells, calls this cell's acceptance real in passing; the page's note, which names the setting, is followed here, and had the acceptance been real it would have been measured on the benchmark's random prompts, on which the same pull request's real drafter accepted 7.2 to 7.4 of 8 on MI350X, an acceptance it says real text will not reach. With DSPARK the KDA state pool clamps admission at 68 running requests. Speculation emits more than one token per decode step, so the bandwidth check, which assumes one, overstates what this rate demands of memory. Decode and prefill split that one run's tokens at its own shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which reproduces the run only at 8,192 tokens in and 1,024 out. At another prompt share and the same request length it understates, because one server can move GPU time between the two phases and the calculator holds each limit fixed. For requests longer than 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the run. The run's random prompts shared no prefixes, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from this system than the calculator credits. [CORRECTED 2026-09-13: this profile was 22,500 decode and 225,000 prefill tok/s, SGLang's 2,808 tok/s per GPU read as decode throughput and scaled to 8 accelerators, with prefill set at ten times decode. That figure counts prompt tokens as well, divides by the 16 GPUs of a one-prefill, one-decode arm, and comes from a sweep tagged fp4 that the post never defines.]
The SGLang cookbook's MI350X Balanced cell at concurrency 16: TP8 on ROCm with AITER's A8W4 MoE kernels, Triton attention, an fp8_e4m3 cache where the calculator defaults to bf16, and no speculation, 462 tok/s per GPU on requests of 8,192 prompt and 1,024 generated tokens. That figure counts both kinds of token, so the eight GPUs served 3,696 tok/s, 411 decode tok/s generated and 3,285 prefill tok/s of prompt, and the cell's medians agree: 16 × 1,024 / (6.331 + 1,024 × 0.0328) = 410.4 generated tok/s. Measured on MI350X and carried to the MI355X, the part this catalog carries. The MI355X has the MI350X's 288 GB of HBM3E per GPU at 8 TB/s, by AMD's pages for both, and adds compute: 10.1 PFLOPs of peak MXFP4 performance against 9.2 and a 2,400 MHz peak engine clock against 2,200, at 1,400 W typical board power against 1,000, and the cookbook serves both from one recipe and one image. Decode is usually bounded by memory bandwidth, which the two share, and prefill by compute, where the MI355X has about a tenth more, so these figures more likely understate an MI355X than overstate it, by up to about a tenth where compute binds. Graded low because the cookbook marks every MI350X cell Final Verification In Progress and publishes no speed round for the MI355X. Decode and prefill split that one run's tokens at its own shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which reproduces the run only at 8,192 tokens in and 1,024 out. At another prompt share and the same request length it understates, because one server can move GPU time between the two phases and the calculator holds each limit fixed. For requests longer than 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the run. The run's random prompts shared no prefixes, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from this system than the calculator credits. [UPGRADED 2026-09-13 from an assumption. The profile was 1,000 decode and 15,000 prefill tok/s, and its note read "Still a pure planning assumption. AMD published a day-0 Kimi K3 bring-up on MI355X but deliberately reported correctness only, with no throughput, TTFT or TPOT figures, so there is nothing to anchor these numbers to."]
The SGLang cookbook's MI350X Balanced cell at concurrency 64, on the recipe of the concurrency-16 cell: 813 tok/s per GPU on requests of 8,192 prompt and 1,024 generated tokens. That figure counts both kinds of token, so the eight GPUs served 6,504 tok/s, 723 decode tok/s generated and 5,781 prefill tok/s of prompt, and the cell's medians agree: 64 × 1,024 / (18.665 + 1,024 × 0.07039) = 722.2 generated tok/s. SGLang pull request #39029, which added the cell, gives the running-request cap at the run's memory budget as 74, above the run's 64. Measured on MI350X and carried to the MI355X, the part this catalog carries. The MI355X has the MI350X's 288 GB of HBM3E per GPU at 8 TB/s, by AMD's pages for both, and adds compute: 10.1 PFLOPs of peak MXFP4 performance against 9.2 and a 2,400 MHz peak engine clock against 2,200, at 1,400 W typical board power against 1,000, and the cookbook serves both from one recipe and one image. Decode is usually bounded by memory bandwidth, which the two share, and prefill by compute, where the MI355X has about a tenth more, so these figures more likely understate an MI355X than overstate it, by up to about a tenth where compute binds. Graded low because the cookbook marks every MI350X cell Final Verification In Progress and publishes no speed round for the MI355X. Decode and prefill split that one run's tokens at its own shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which reproduces the run only at 8,192 tokens in and 1,024 out. At another prompt share and the same request length it understates, because one server can move GPU time between the two phases and the calculator holds each limit fixed. For requests longer than 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the run. The run's random prompts shared no prefixes, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from this system than the calculator credits. [UPGRADED 2026-09-13 from an assumption. The profile was 2,000 decode and 30,000 prefill tok/s, and its note read "Planning assumption, not a benchmark."]
The SGLang cookbook's MI350X Balanced DSPARK cell at concurrency 64, the same recipe with DSPARK speculative decoding: 913 tok/s per GPU on requests of 8,192 prompt and 1,024 generated tokens. That figure counts both kinds of token, so the eight GPUs served 7,304 tok/s, 812 decode tok/s generated and 6,492 prefill tok/s of prompt. The cell's acceptance was pinned, not measured: SGLang pull request #39029, which added it, ran DSPARK with SGLANG_SIMULATE_ACC_LEN=5, which accepts exactly 5 of the 8 draft tokens every round. So these rates are what the system delivers when that share of draft tokens is accepted, not what DSPARK delivers on any workload: one whose drafts are accepted less often gets less, and the cookbook says no measured acceptance rate for a real workload exists yet. The pull request left out a run with the real drafter, which accepted 7.2 to 7.4 of 8 on the benchmark's random prompts, as an acceptance real text will not reach. This is the one cell whose medians do not close: 64 × 1,024 / (23.720 + 1,024 × 0.03145) = 1,171.9 generated tok/s, against 811.6 from the per-GPU figure. The profile takes the per-GPU figure, a whole-run total and the lower reading, while the same recipe's concurrency-16 DSPARK cell, 864 tok/s per GPU, agrees with its medians within 2%. The pull request gives the running-request cap with DSPARK at the run's memory budget as 48, so at most 48 of the run's 64 requests ran at once, and calls the concurrency-64 cells partly queue-bound. Measured on MI350X and carried to the MI355X, the part this catalog carries. The MI355X has the MI350X's 288 GB of HBM3E per GPU at 8 TB/s, by AMD's pages for both, and adds compute: 10.1 PFLOPs of peak MXFP4 performance against 9.2 and a 2,400 MHz peak engine clock against 2,200, at 1,400 W typical board power against 1,000, and the cookbook serves both from one recipe and one image. Decode is usually bounded by memory bandwidth, which the two share, and prefill by compute, where the MI355X has about a tenth more, so these figures more likely understate an MI355X than overstate it, by up to about a tenth where compute binds. Graded low because the cookbook marks every MI350X cell Final Verification In Progress and publishes no speed round for the MI355X, and because the acceptance behind this case was pinned rather than measured. Speculation emits more than one token per decode step, so the bandwidth check, which assumes one, overstates what this rate demands of memory. Decode and prefill split that one run's tokens at its own shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which reproduces the run only at 8,192 tokens in and 1,024 out. At another prompt share and the same request length it understates, because one server can move GPU time between the two phases and the calculator holds each limit fixed. For requests longer than 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the run. The run's random prompts shared no prefixes, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from this system than the calculator credits. [UPGRADED 2026-09-13 from an assumption. The profile was 4,000 decode and 60,000 prefill tok/s, and its note read "Optimistic planning assumption, not a benchmark."]
Nine unified eight-GPU servers across the rack's 72 GPUs, each at the SGLang cookbook's B300 Balanced cell, 1,395 tok/s per GPU counting prompt and generated tokens: 9 × 8 × 1,395 = 100,440 tok/s, 11,160 decode tok/s generated and 89,280 prefill tok/s of prompt at 8,192 tokens in and 1,024 out. A GB300 carries the B300 GPU, and this case uses none of what the rack adds: no prefill/decode disaggregation and no NVLink domain wider than eight GPUs. The cell ran on a B300 1×8 node at concurrency 64, not on GB300, so it records no concurrency for the rack. Decode and prefill split that one run's tokens at its own shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which reproduces the run only at 8,192 tokens in and 1,024 out. At another prompt share and the same request length it understates, because each server can move GPU time between the two phases and the calculator holds each limit fixed. For requests longer than 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the run. The run's random prompts shared no prefixes, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from these servers than the calculator credits. [CORRECTED 2026-09-13: this profile was 95,000 decode and 950,000 prefill tok/s, SGLang's 2,633 tok/s per GPU read as decode throughput on 8 GB300, multiplied by 72 at an assumed 50% scaling efficiency, with prefill set at ten times decode. That figure counts prompt tokens as well and divides by the 32 GPUs of a two-prefill, two-decode arm.]
Two copies of the DCP composition from SGLang's Kimi K3 post, two PP8 prefill workers feeding two DCP8 decode nodes on 32 GPUs at 2,633 tok/s per GPU, fill 64 of the rack's 72 GPUs, and one unified server at the cookbook's B300 Balanced cell, 1,395 tok/s per GPU, fills the other 8: 2 × 32 × 2,633 + 8 × 1,395 = 179,672 tok/s counting prompt and generated tokens, 19,964 decode tok/s generated and 159,708 prefill tok/s of prompt. Both runs split their tokens 8:1, the post's at 8,000 in and 1,000 out and the cookbook's at 8,192 and 1,024. The post's per-GPU figures divide by every GPU of an arm, prefill workers included, which the post does not state and its source record infers from the post's own measurements. Replication is the assumption: the post's 2P:2D arm at 2,633 sits within 2% of the single PP8 → DCP8 arm's 2,682.7, a gap its caption calls indistinguishable, but no published Kimi K3 serving run spans the rack, and the largest arm on the frontier uses 48 GPUs. The post prints no concurrency for either arm. The arms' prefill and decode workers, 64 of the rack's 72 GPUs, are sized for that 8:1 shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which is what fixed workers give: a deployment serving another prompt share would re-divide the rack between prefill and decode workers and reach more than the calculator credits. For requests longer than the runs' 9,000 to 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the runs. The post does not describe its prompts, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from this rack than the calculator credits. [CORRECTED 2026-09-13: this profile was 130,000 decode and 1,300,000 prefill tok/s, the same 2,633 tok/s per GPU read as decode throughput on 8 GB300, multiplied by 72 at an assumed 70% scaling efficiency, with prefill set at ten times decode.]
Four copies of the highest untagged arm on SGLang's Kimi K3 serving frontier, one PP8 prefill worker feeding one TP8 decode node on 16 GPUs at 2,715 tok/s per GPU, fill 64 of the rack's 72 GPUs, and one unified server at the cookbook's B300 Balanced cell, 1,395 tok/s per GPU, fills the other 8: 4 × 16 × 2,715 + 8 × 1,395 = 184,920 tok/s counting prompt and generated tokens, 20,547 decode tok/s generated and 164,373 prefill tok/s of prompt. The arm of the same topology tagged fp4 peaks higher, at 2,808, and would make this 190,872; it is left out because the post never defines fp4, and a rate does not carry across a precision the catalog cannot name. The post's per-GPU figures divide by every GPU of an arm, prefill workers included, which the post does not state and its source record infers from the post's own measurements. Replication is the assumption, and a larger one than in the base case: four copies where the post measured at most two, and no published Kimi K3 serving run spans the rack. The arms' prefill and decode workers, 64 of the rack's 72 GPUs, are sized for that 8:1 shape, and the calculator costs a scenario at whichever of the two limits it reaches first, which is what fixed workers give: a deployment serving another prompt share would re-divide the rack between prefill and decode workers and reach more than the calculator credits. For requests longer than the runs' 9,000 to 9,216 tokens it overstates, because each prompt token then attends over, and each decode step reads, more cache than in the runs. The post does not describe its prompts, and the calculator models no prefix cache: it charges every input token as prefill, so a workload that reuses prefixes gets more from this rack than the calculator credits. [CORRECTED 2026-09-13: this profile was 180,000 decode and 1,800,000 prefill tok/s, SGLang's 2,808 tok/s per GPU read as decode throughput on 8 GB300, multiplied by 72 at an assumed 90% scaling efficiency, with prefill set at ten times decode. That figure counts prompt tokens as well and divides by the 16 GPUs of a one-prefill, one-decode arm.]
Published by the vLLM team about its own engine on day zero. A real measurement, but self-reported by the party with an interest in the result, and not yet independently reproduced.
Published by the vLLM team about its own engine on day zero; not independently reproduced.
Published by the SGLang/LMSYS team about its own engine on day zero. Reproducing it requires running the same disaggregated prefill/decode split, not a default single-server deployment.
Published by the SGLang/LMSYS team about its own engine on day zero. Reproducing it requires running the same disaggregated prefill/decode split, not a default single-server deployment. The figure is read from the chart's data coordinates rather than quoted from its text.
Published by the SGLang/LMSYS team about its own engine on day zero. Reproducing it requires running the same disaggregated prefill/decode split, not a default single-server deployment.
Published by the SGLang/LMSYS team about its own engine; not independently reproduced. [DOWNGRADED 2026-09-13 from medium: the figure the post cites for 541 tok/s implies about 427.]
Published by the SGLang project about its own engine; not independently reproduced.
Published by the SGLang project about its own engine; not independently reproduced.
Published by the SGLang project about its own engine; not independently reproduced. Graded low because the acceptance length behind the figure is simulated, by the page's own note; a pull request of the project's says otherwise in passing, and the row's notes give both.
Published by the SGLang project about its own engine, before the cookbook's final verification round on the released weights.
Published by the SGLang project about its own engine, before the cookbook's final verification round on the released weights.
Published by the SGLang project about its own engine, before the cookbook's final verification round on the released weights. Graded low for that, and because the acceptance behind the figure was pinned rather than measured: the run timed the engine at an acceptance chosen in advance.
Token-weighted mean of 64,680 individual API requests across 912 threads recorded between 2026-06-10 and 2026-08-16. The input mix measured 97.97% cache hit, 1.99% cache write and 0.037% uncached; rounding to 98/2/0 is deliberate, because giving the uncached bucket a token 1% is LESS accurate than zero when uncached input costs ten times a cache hit. An independent Claude Code user publishing a 30-day aggregate lands at 96.58/3.35/0.07, within 1.4 points. Caveat worth carrying: this is one person’s working style, with unusually long sessions — public figures put typical cache-write shares at 3-7% rather than 2%, which is exactly what the session-length curve predicts.
Derived, not measured. A conversation of N turns over a stable prefix bills 2/(N+1) of its input as cache writes; at a dozen turns that is 15%. Corroborated by the shortest real sessions in the recorded corpus, which measure 83.84% hit and 14.90% write. The token counts are judgement.
An assumption with its arithmetic stated: one document written to cache once and re-read for each of five follow-up questions gives a 20% write share. Change the question count and the mix moves a long way — two questions gives 50%, twenty gives 5% — so this is the number in the catalog most worth overriding. At a single question caching cannot pay for itself at all and the honest mix is none.
The same shape as long-document reading, assuming twelve questions rather than five because a million-token write is worth amortising further. Same caveat: the question count is the assumption doing the work.
A short brief against a shared style guide. Assumes roughly 60% of the prompt is a cacheable shared prefix. Output dominates the bill at this shape, so the input mix barely moves the answer — which is why a low-confidence assumption is tolerable here and would not be in the agentic preset.