Temperature zero is not a determinism guarantee

One prompt, temperature zero, 1,000 completions: 80 different outputs. Horace He and colleagues at Thinking Machines Lab reported that result for Qwen3-235B-A22B-Instruct-2507 in non-thinking mode. Then they enabled batch-invariant kernels and obtained 1,000 identical completions.

Both halves matter. The first defeats the promise that a sampling setting guarantees repeatability. The second defeats the idea that deterministic inference is impossible, or that a cache is the only way to obtain it.

Their September 2025 investigation explains how changing batch sizes can change floating-point reduction paths. Greedy selection can then choose a different token because the numbers reaching the sampler differ. That is a demonstrated mechanism in the studied stack, not a diagnosis of every hosted API or every inconsistent application. These are the authors' results; I haven't reproduced their inference experiment.

Name the thing that stays the same

My rule for a repeatability promise is to make the team finish this sentence: “The same inputs produce the same ___.” Exact text, a parsed score and a business decision are different commitments. A system can satisfy one and fail another.

As an illustration, consider a pitch scorer that retrieves pay-range research, sends that evidence to a model and maps the response to pass, caution or fail. The research lookup might come from a database while the verdict is generated afresh. Calling that application “cached” tells the reviewer very little.

I would separate the review into three checks:

• Research reuse: which external facts were reused, under what key and from what date?
• Fresh inference: which model, request settings, prompt and evidence produced this response?
• Decision replay: was the final verdict recomputed, or was an existing result returned unchanged?

Keeping those checks separate prevents a common review shortcut: treating evidence that one stage repeated as proof that the whole application repeated. It also gives the failure somewhere specific to land. A changed source document calls for a different investigation from a changed answer to an otherwise fixed request.

A saved answer is a different contract

For an application that needs to replay a decision, I would store the accepted result with the inputs and policy version that gave it meaning. The proposed contract is then explicit: return this recorded decision for this recorded case. It is not a promise to regenerate identical text from a model.

That choice needs an invalidation policy. In the pitch-scorer illustration, a changed pay range or scoring rule could make yesterday's verdict inappropriate today. A time-to-live can be one control, but I would also account for known source and policy changes rather than wait for a timer. Repeating an answer and justifying it are separate jobs.

For fresh inference, I would ask the serving team what reproducibility they actually support and under which configuration. Don't diagnose batch sensitivity from a screen recording of two different answers. Preserve the request, retrieved inputs and version information first; otherwise the comparison may not even hold the application inputs fixed.

Test the promise you intend to sell

Before signing off, I would run repeated requests against a fixed test set and compare the exact output, parsed fields and final decision separately. Then repeat the exercise after deliberately changing one dependency, such as a source snapshot or scoring rule. This is a proposed test plan, not a result from a client deployment.

The useful report should distinguish observed repeatability from a supported guarantee. A finite run without differences is evidence about those runs; it isn't proof that every future request will match. Conversely, one difference is enough to reject an unconditional exact-output promise for the tested configuration.

Temperature zero is a setting. Repeatability is a contract: name what repeats, preserve its inputs and test the boundary that promises it.