41B active parameters still need a 600 GB inference cluster
Inkling activates 41B of 975B parameters per token, yet its NVFP4 checkpoint needs 600 GB of aggregate VRAM. Active parameters size arithmetic—not resident weights, cache or topology.
Thinking Machines released Inkling with a number that looks like a small-cluster model: 41 billion active parameters. The same release says the mixture-of-experts checkpoint contains 975 billion parameters. Then Hugging Face's deployment guide supplies the number a capacity plan actually needs: 600 GB of aggregate VRAM for NVFP4, or 2 TB for BF16.
There is no contradiction. Active parameters describe how much of the model participates in one token's computation. They do not describe how much model must be available for the router to choose from.
That distinction is nearly a 24-fold planning error if it lands in the wrong spreadsheet cell.
The router saves arithmetic, not inventory
Inkling's July 15 release describes a 975B-parameter model with 256 routed experts, six selected experts and two shared experts active per token. The sparse route is the efficiency mechanism: one token does not traverse every expert.
But tomorrow's token may choose a different route. The serving system still needs access to the full checkpoint, spread across the deployment. This is why the Hugging Face guide pinned to its July 31 revision can put “41B active” and “600 GB aggregate VRAM” on the same page.
The rough weight arithmetic makes the gap visible. If 41B were the storage number, BF16 would suggest about 82 GB. The full 975B checkpoint at two bytes per parameter is about 1.95 TB before runtime headroom—almost exactly the guide's 2 TB figure. At four bits, the raw lower bound is about 487.5 GB; the published NVFP4 deployment requirement is 600 GB.
Those calculations are orientation, not a purchase order. Some tensors, quantisation metadata and serving structures do not fit the one-number shortcut. The useful point is simpler: total parameters explain residency; active parameters explain a large part of per-token arithmetic.
One model needs three capacity budgets
I would reject any capacity sheet with one column called model size. For a sparse model, it hides three different constraints:
- Residency: checkpoint bytes at the chosen precision, sharding overhead and memory left after loading.
- Compute: active parameters per token, routing balance and the measured throughput of the actual serving engine.
- Runtime state: KV cache at the context and concurrency you will permit, plus batching and multimodal inputs.
Inkling makes the third budget hard to ignore because it supports context up to one million tokens. Hugging Face's own serving notes tell operators to leave static-memory headroom for KV cache and cap maximum model length when cache pressure bites. A checkpoint fitting across the cards is therefore the start of the plan, not the pass condition.
The topology belongs on the same page. “600 GB aggregate” means the weights are sharded, so GPU count, interconnect and the failure of one rank become product constraints. The launch material recommends four B300 or GB200 GPUs for the NVFP4 configuration and explicitly calls for multi-node interconnect in one of the BF16 configurations. Neither source publishes an independent production-throughput study for those clusters.
Put the right number in the approval memo
The earlier AndyMental rule was that local-model benchmarks are not capacity plans. Sparse models add a more basic trap: the number in the model headline may not even be the number that loads into memory.
Steal this five-column check before reserving hardware: total checkpoint bytes at the exact quantisation; active parameters per token; aggregate usable VRAM after serving overhead; allowed context times target concurrency; and measured p95 throughput on the proposed topology. Keep vendor-recommended hardware clearly labelled until your own load test replaces it.
My release gate is blunt: if the memo uses active parameters to size residency, it is not ready for a GPU quote. Sparsity can make a huge model cheaper to run per token. It does not make the other experts vanish from the machine.
Size storage from the weights that must be available, compute from the weights that activate, and the cluster from both.
