| Accuracy | 95.1% (95% CI 93.5–96.7%), over 2,397 questions |
|---|---|
| Decision-zone calibration gap (60%–90% confidence) | 12.3 pts (95% CI 4.2 to 22.3 pts) |
| Signed gap (accuracy minus confidence) | -12.3 pts (95% CI -22.3 to -2.2 pts): overconfident |
| Answers in the decision zone | 224 (9.3% of all answers) |
| At threshold | default 70% — none documented: accuracy 96.2% (95% CI 94.7–97.5%), coverage 98.0% (95% CI 97.2–98.9%) — it was sure enough to act on 2,350 of 2,397 |
On the answers given with each level of confidence, how often the model was right. A well-calibrated model is right about as often as it says.
| Stated confidence | Answers | Mean confidence | Actually right |
|---|---|---|---|
| 60%–70% | 39 | 63.5% | 48.7% |
| 70%–80% | 64 | 73.6% | 64.1% |
| 80%–90% | 121 | 82.4% | 69.4% |
| Expected calibration error (secondary) | 0.021 (95% CI 0.009–0.036). ECE upper 95% bound 0.036 ≤ 0.05: consistent with the calibration claim |
|---|---|
| Brier score (lower is better) | 0.035 |
| No answer given | 0.0% |
| Same probability on every repeat | 73.7% of 859 repeated questions |
| Cost per 1,000 answers | — (not hosted) |
| Measured under | Jev study (closed) — general-purpose reference |
No paired comparison with the leader is reported for this row.
Data generated November 22, 2026. We publish the question, never the trap: no question, answer key or raw response is shown. How we measure.
Start with the question you actually arrived with — there are five: