A decision model answers a yes/no question with a single probability instead of text. That makes it a natural gatekeeper in front of an AI agent: should this command run, is this task really finished, does this message need a person? Decision models are cheap to run, and 7 of the 8 we measured gave exactly the same probability every time they were asked the same question, so a team can write a rule such as “act only when it is 90% sure, otherwise ask a person” and audit it afterwards. On our questions the hosted ones cost 2.6–4.8 cents per 1,000 answers.
The catch is that every one of them is sold on the claim that its probability can be trusted, and until we measured them, every published figure was the vendor’s own.
When not to use one: when being wrong is costly and the volume is low. On our questions the general-purpose models we ran as a reference averaged 94.9% accuracy, about level with the best decision model (94.7%), and more accurate than the rest.
Sorted by accuracy. The decision-zone gap is how far a model’s stated confidence was from how often it was actually right, on the answers it gave with 60%–90% confidence — where a person or a rule has to act. Smaller is better. The direction is named only when the 95% interval rules out zero. At threshold shows accuracy and coverage (how often the model was sure enough to act) at the vendor’s documented threshold, or a default of 70% where none is documented.
| # | Model | Where it runs | Accuracy95% CI | Decision-zone gap95% CI | Direction | At thresholdaccuracy / coverage | Measured | Downloadspopularity, not quality |
|---|---|---|---|---|---|---|---|---|
| 1 | pplx-decider v1.1 27BPerplexity | Self-hosted (data-center GPU) | 94.7%93.1–96.4% | 14.8 pts7.6 to 25.6 pts | no clear directionsigned -9.3 pts | 95.4% / 97.2%default 70% — none documented | 2026-10-11 | 36,407Perplexity pplx-decider family, 30 days to 2026-11-19 |
| 2 | OpenAI DecisionsOpenAI | Maker's API | 88.0%85.6–90.2% | 1.6 pts1.3 to 8.9 pts | no clear directionsigned +0.2 pts | 91.3% / 89.6%default 70% — none documented | 2026-10-11 | — |
| 3 | JevTypeSafe | Maker's API | 86.4%84.0–88.7% | 7.4 pts4.0 to 11.1 pts | underconfidentsigned +7.4 pts | 94.6% / 74.7%vendor's documented threshold, 70% | 2026-10-11 | — |
| 4 | OpenDecider smallindependent developer | Self-hosted | 84.0%81.1–86.5% | 9.6 pts6.5 to 12.4 pts | underconfidentsigned +9.6 pts | 92.0% / 70.1%default 70% — none documented | 2026-10-11 | 17,861OpenDecider family (incl. community GGUF), 30 days to 2026-11-19 |
| 5 | Gemini DecisionsGoogle | Maker's API | 82.5%79.9–85.2% | 10.7 pts5.7 to 16.7 pts | overconfidentsigned -10.7 pts | 90.3% / 82.5%default 70% — none documented | 2026-11-19 | — |
| 6 | GLiNER2.5-DecideFastino | Self-hosted | 69.7%66.8–72.7% | 5.8 pts3.8 to 9.3 pts | underconfidentsigned +5.6 pts | 83.8% / 54.8%default 70% — none documented | 2026-10-11 | 81,604Fastino GLiNER2.5-Decide family, 30 days to 2026-11-19 |
| 7 | strands-decider 2BAWS (Strands Agents) | Self-hosted | 66.3%62.8–69.8% | 11.6 pts8.4 to 14.6 pts | no clear directionsigned -1.4 pts | 87.9% / 51.6%default 70% — none documented | 2026-10-11 | — |
| 8 | LayaConvai Innovations | Self-hosted | 64.8%61.5–68.1% | 5.5 pts2.3 to 9.3 pts | overconfidentsigned -5.5 pts | 73.1% / 48.9%default 70% — none documented | 2026-10-11 | 27,918Convai Laya family, 30 days to 2026-11-19 |
Downloads are popularity, not quality: each count covers the whole maker family named beside it, over the window shown, read on the date shown. Hosted models have no download count. Select a model for its full numbers.
General-purpose models asked the same questions, for comparison. They are a reference, not entries in the battery, and are not ranked.
| Reference | Where it runs | Accuracy95% CI | Decision-zone gap95% CI | Direction | At thresholdaccuracy / coverage | Measured |
|---|---|---|---|---|---|---|
| DeepSeek V4 FlashJev study, collected 2026-10-09/10 | — | 94.7%93.1–96.1% | 35.7 pts22.8 to 51.1 pts | overconfidentsigned -34.3 pts | 94.7% / 100.0%default 70% — none documented | 2026-10-10 |
| GPT-6.1 SolJev study, collected 2026-10-09/10 | — | 95.1%93.5–96.7% | 12.3 pts4.2 to 22.3 pts | overconfidentsigned -12.3 pts | 96.2% / 98.0%default 70% — none documented | 2026-10-10 |
The 4 models with a public download count, by accuracy and by downloads. Downloads are on a log scale and count each maker’s whole family.
The full method, in plain English: How the Decision Model Battery works.
8 separate verdicts are reported without correction: at 95% each, the chance that at least one lands on the wrong side by chance alone is about 34%.
Start with the question you actually arrived with — there are five: