Rank 1 of 8 by accuracy

pplx-decider v1.1 27B

PerplexitySelf-hosted (data-center GPU)Measured 2026-10-11Protocol 1.0

Headline

Accuracy94.7% (95% CI 93.1–96.4%), over 2,391 questions
Decision-zone calibration gap (60%–90% confidence)14.8 pts (95% CI 7.6 to 25.6 pts)
Signed gap (accuracy minus confidence)-9.3 pts (95% CI -19.4 to 0.6 pts): no clear direction
Answers in the decision zone243 (10.2% of all answers)
At thresholddefault 70% — none documented: accuracy 95.4% (95% CI 93.7–96.9%), coverage 97.2% (95% CI 96.0–98.3%) — it was sure enough to act on 2,325 of 2,391

Decision zone, bin by bin

On the answers given with each level of confidence, how often the model was right. A well-calibrated model is right about as often as it says.

Stated confidenceAnswersMean confidenceActually right
60%–70%2764.5%88.9%
70%–80%8775.9%72.4%
80%–90%12985.5%65.1%

Other measures

Expected calibration error (secondary)0.026 (95% CI 0.014–0.044). ECE upper 95% bound 0.044 ≤ 0.05: consistent with the calibration claim
Brier score (lower is better)0.044
No answer given0.0%
Same probability on every repeat100.0% of 859 repeated questions
Cost per 1,000 answers— (not hosted)
Downloads (popularity, not quality)36,407 — Perplexity pplx-decider family, 30 days, read 2026-11-19
Modelperplexity-ai/pplx-decider-v1.1-27b @ 6195aa55a72a
Measured underdecision-models study (closed)

Compared with the leader

This is the current leader by accuracy.

Data generated November 22, 2026. We publish the question, never the trap: no question, answer key or raw response is shown. How we measure.