| Accuracy | 69.7% (95% CI 66.8–72.7%), over 2,397 questions |
|---|---|
| Decision-zone calibration gap (60%–90% confidence) | 5.8 pts (95% CI 3.8 to 9.3 pts) |
| Signed gap (accuracy minus confidence) | +5.6 pts (95% CI 2.2 to 8.8 pts): underconfident |
| Answers in the decision zone | 1,536 (64.1% of all answers) |
| At threshold | default 70% — none documented: accuracy 83.8% (95% CI 80.4–87.0%), coverage 54.8% (95% CI 51.3–58.3%) — it was sure enough to act on 1,314 of 2,397 |
On the answers given with each level of confidence, how often the model was right. A well-calibrated model is right about as often as it says.
| Stated confidence | Answers | Mean confidence | Actually right |
|---|---|---|---|
| 60%–70% | 381 | 65.7% | 65.4% |
| 70%–80% | 681 | 74.5% | 83.3% |
| 80%–90% | 474 | 85.3% | 91.1% |
| Expected calibration error (secondary) | 0.082 (95% CI 0.058–0.115). ECE lower 95% bound 0.058 > 0.05: calibration claim not supported on these items |
|---|---|
| Brier score (lower is better) | 0.191 |
| No answer given | 0.0% |
| Same probability on every repeat | 100.0% of 859 repeated questions |
| Cost per 1,000 answers | — (not hosted) |
| Downloads (popularity, not quality) | 81,604 — Fastino GLiNER2.5-Decide family, 30 days, read 2026-11-19 |
| Model | fastino/GLiNER2.5-Decide @ 9d1bfb848cd1 |
| Measured under | decision-models study (closed) |
On the 2,391 questions both answered, GLiNER2.5-Decide was 24.8 pts less accurate than pplx-decider v1.1 27B (paired difference -24.8 pts, 95% CI -28.3 to -21.4 pts).
Data generated November 22, 2026. We publish the question, never the trap: no question, answer key or raw response is shown. How we measure.
Start with the question you actually arrived with — there are five: