Model Calibration
Prediction Accuracy and Calibration
Perfect calibration means a 70% confidence pick wins 70% of the time. See how we measure up across 241 settled predictions.
241
Settled Predictions
0.2541
Brier Score (lower is better)
5
Confidence Buckets
Predicted vs Actual Win Rate
Calibration by Confidence Band
| Confidence Band | Predicted % | Actual % | Delta | Sample Size |
|---|---|---|---|---|
| 50-54% | 52.0% | 47.6% | -4.4% | 21 |
| 55-59% | 57.0% | 55.3% | -1.7% | 114 |
| 60-64% | 62.0% | 53.3% | -8.7% | 45 |
| 65-69% | 67.0% | 46.7% | -20.3% | 45 |
| 70-74% | 72.0% | 75.0% | +3.0% | 16 |
What is calibration? A well-calibrated model's stated confidence matches reality. If we say 80% confidence, those picks should win about 80% of the time.
What is the Brier Score? It measures the accuracy of probabilistic predictions. A score of 0 is perfect; 0.25 is no better than a coin flip. Lower is better.
The diagonal line on the chart represents perfect calibration. Points above the line mean we're underconfident (actual wins exceed predicted); points below mean we're overconfident.