LLM evaluation · Uncertainty

The free uncertainty signal kept up with the expensive one

I wanted to know whether a small local LLM can tell when it is about to be wrong, and how much that knowledge costs. The method everyone reaches for, semantic entropy, takes ten extra generations and an NLI model. A number you get for free from the answer you were already generating did about as well. The interesting part is why.

The question

If a model gives you an answer, can a cheap signal rank its wrong answers above its right ones, and is the expensive signal worth paying for? That is the practical question behind every "should this answer be shown, flagged or escalated?" decision.

The setup

Code, raw per-question records and the generated report are in llm-uncertainty-quantification.

The result

Task (test split)Best free signalSemantic entropyBest sampled signal
NQ-Open100 questions0.94595% CI 0.899–0.9810.90095% CI 0.826–0.9560.916lexical entropy
GSM8K50 problems0.82895% CI 0.684–0.9450.79495% CI 0.659–0.9110.884lexical entropy

On factual questions, sequence log-probability from the single greedy pass came out on top. On maths, signals built from the samples led. In both tasks the confidence intervals overlap, so the honest reading is a tie on factual QA, not a win. A tie is still the headline: the free signal costs nothing beyond the answer you were already generating, while the expensive one costs ten more generations and an NLI pass.

Bar chart of error-detection AUROC on NQ-Open. Sequence log-prob leads at 0.945, lexical entropy 0.916, semantic entropy 0.900, with overlapping 95% intervals.
Error-detection AUROC on NQ-Open, test split. Orange bars are free single-pass signals, blue bars need sampling. The legend overlaps the last bar in the original figure.

The surprise: semantic clustering did not help

Look at the blue bars again. Semantic entropy (0.900) did not beat lexical entropy (0.916), which is the same calculation on exact answer strings, with no NLI at all. The gap is within noise, but it points the wrong way: semantic clustering exists to merge answers like "Paris" and "Paris, France" so that harmless rewording does not look like uncertainty, and that should have helped. On GSM8K it trailed by more (0.794 against 0.884).

The per-question records suggest why. By design the NLI step can only merge distinct answer strings, never split them, and across all 200 NQ-Open questions (dev and test together) it merged them in 87. Some merges are exactly what you want: "neuron" and "neurons", "two years" and "2 years". Many are not:

QuestionWhat the 10 samples saidMeanings found
who has won more national championships michigan or ohio statemichigan ×4, ohio state ×61
what is the baby elephants name in jungle bookbaloo ×9, bagheera ×11
who sings you can't get what you want9 different answers, from Madonna to Guns N' Roses2

When contradictory answers become one meaning, semantic entropy reports the model as certain at exactly the moment it is torn. All three answers above were wrong. That is the worst possible failure for an error detector: confidence on the errors.

The merging is not spread evenly. It hit 54% of "who" questions (47 of 87) but only 11% of "when" questions (4 of 37). My working explanation: each NLI comparison is conditioned on the question, so the two texts being compared are mostly identical and differ only by a name, and a small entailment model reads two near-identical sentences as entailment. It is not only names, though: on GSM8K the same step merged distinct numbers in 18 of 100 problems. I have not isolated the cause with an ablation yet, so treat this as a hypothesis rather than a finding.

Where sampling did earn its cost

On GSM8K, agreement between samples is a strong signal: majority voting over 10 samples lifts accuracy from 0.56 to 0.66. The useful question there is not "free or expensive" but "when to pay". An uncertainty-gated cascade answers greedily, checks the free sequence log-probability, and only samples when that answer looks shaky. The escalation threshold is fitted on the dev split.

Policy (GSM8K test)AccuracyGenerations per question
Greedy only0.561.0
Always sample 10 and vote0.6611.0
Cascade gated on the free signal0.664.8

The cascade escalated 38% of problems and matched always-sampling accuracy with 44% of the generations. With 50 test problems, one problem is worth 0.02 accuracy, so read this as "most of the gain for under half the compute", not as an exact figure.

How I nearly got this wrong. My first version of this cascade was gated on lexical entropy, picked because it had the best AUROC on the test split. That is two mistakes in one: choosing anything by its test score is leakage, and lexical entropy needs the ten samples before it can decide whether to take them, so the "savings" were not real. An independent review caught it. Gating on the free signal with everything fitted on dev turned out no worse, and cheaper.

What I would do in production

Limits

Code, per-question records and the full report: github.com/trohit920/llm-uncertainty-quantification.

← All posts