| Accuracy | 94.7% (95% CI 93.1–96.1%), over 2,282 questions |
|---|---|
| Decision-zone calibration gap (60%–90% confidence) | 35.7 pts (95% CI 22.8 to 51.1 pts) |
| Signed gap (accuracy minus confidence) | -34.3 pts (95% CI -49.3 to -17.1 pts): overconfident |
| Answers in the decision zone | 54 (2.4% of all answers) |
| At threshold | default 70% — none documented: accuracy 94.7% (95% CI 93.1–96.1%), coverage 100.0% (95% CI 99.9–100.0%) — it was sure enough to act on 2,281 of 2,282 |
On the answers given with each level of confidence, how often the model was right. A well-calibrated model is right about as often as it says.
| Stated confidence | Answers | Mean confidence | Actually right |
|---|---|---|---|
| 60%–70% | 1 | too few to show | |
| 70%–80% | 7 | too few to show | |
| 80%–90% | 46 | 82.4% | 41.3% |
| Expected calibration error (secondary) | 0.035 (95% CI 0.023–0.050). ECE 95% CI [0.023, 0.050] spans 0.05: inconclusive |
|---|---|
| Brier score (lower is better) | 0.046 |
| No answer given | 6.9% |
| Same probability on every repeat | 58.5% of 810 repeated questions |
| Cost per 1,000 answers | — (not hosted) |
| Measured under | Jev study (closed) — general-purpose reference |
No paired comparison with the leader is reported for this row.
Data generated November 22, 2026. We publish the question, never the trap: no question, answer key or raw response is shown. How we measure.
Start with the question you actually arrived with — there are five: