Stochastic Outputs
Language models generate text by sampling from probability distributions. This means the same prompt can produce different outputs on different runs — and no configuration setting eliminates this fully.
How text generation works
A language model does not look up an answer or apply a deterministic rule. It predicts, one token at a time, a probability distribution over what should come next. Each step samples from that distribution. A typical response involves thousands of these decisions.
Because each step is a sample, introducing any randomness into a single step changes all subsequent context — and therefore all subsequent decisions. Two runs that diverge on the first token can produce qualitatively different responses by the end.
This is not a bug. It is how the model works, and it is what makes outputs creative and contextually rich. But it also means you cannot evaluate a stochastic system with a single measurement.
Temperature zero is not deterministic
A widespread belief is that setting temperature=0 makes a language model fully deterministic — that the same input will always produce the same output. This is false.
A 2025 study ("Non-Determinism of Deterministic LLM Settings") found that popular models are "rarely deterministic at the raw output level" even with temperature set to zero. A separate study showed that changing only the evaluation batch size — while keeping temperature at zero — produced measurably different outputs.
The root cause is the non-associative property of floating-point arithmetic. GPU computations happen in parallel and their order can vary. Tiny numerical differences accumulate. When two candidate tokens have probabilities very close to each other, these differences can flip which token is selected. Once one token differs, everything downstream diverges.
Research finding
Running a prompt 1,000 times at temperature zero on a modern large model produced 80 unique completions. The most common completion occurred only 78 out of 1,000 times. Source: Thinking Machines Lab (2025).
Practical consequences
Single evaluations are unreliable estimators
A single evaluation run gives you one sample from a distribution. It might be a fortunate run from a system that fails half the time, or an unfortunate run from a system that's excellent most of the time. You don't know which.
Score precision is illusory
An evaluation report saying "your AI scored 85%" is a point estimate with no uncertainty bounds. Research has shown that such estimates might represent anywhere from 70% to 98% depending on how many samples were taken. A confidence interval is essential for any score to be meaningful.
Distributions can be multimodal
Your AI might not produce a single type of response with some variance. It might produce two qualitatively different response types, switching between them randomly. A single mean score hides this entirely. See Behavioral Modes.
What this means for evaluation
Evaluating a stochastic system correctly requires treating the AI's outputs as a statistical population, not as individual data points. The right questions are:
- →What fraction of runs pass the quality threshold? (pass rate, not just pass/fail)
- →How confident are we in that fraction? (confidence interval)
- →Is the AI producing one type of response or multiple? (mode detection)
- →Has the distribution changed compared to the last known-good version? (shift detection)
These are the questions PromptEval is designed to answer. See Why Single Evals Fail for a deeper treatment of the statistical problem.