Model Performance
Every FINAL game is scored against each model's pre-game prediction, built only from rating snapshots dated before tip-off. The Book Closing Line row is the sportsbook benchmark. Model predictions are for analytical purposes only and do not constitute betting advice.
Spread predicted home margin vs. actual
| Model | Games | MAE | RMSE | Winner % |
|---|---|---|---|---|
| Baseline (ML) | 5,531 | 9.36 | 11.90 | 70.7% |
| Baseline Plus (ML) | 5,425 | 9.28 | 11.79 | 70.6% |
| Baseline Tuned (ML) | 5,531 | 9.40 | 12.00 | 70.9% |
| Efficiency Plus (ML) | 5,293 | 9.17 | 11.66 | 70.5% |
| Massey | 5,531 | 9.53 | 12.10 | 70.4% |
| Adjusted Efficiency | 5,531 | 9.52 | 12.10 | 70.3% |
| Book Closing Line | 5,705 | 8.92 | 11.23 | 72.7% |
Total predicted combined score vs. actual
| Model | Games | MAE | RMSE | Bias |
|---|---|---|---|---|
| Baseline (ML) | 5,531 | 14.18 | 17.97 | +4.39 |
| Baseline Plus (ML) | 5,425 | 13.90 | 17.63 | +3.70 |
| Baseline Tuned (ML) | 5,531 | 14.27 | 18.12 | +5.08 |
| Efficiency Plus (ML) | 5,293 | 13.94 | 17.69 | +3.93 |
| Massey Totals | 5,531 | 14.10 | 17.74 | -1.28 |
| Adjusted Efficiency | 5,531 | 14.13 | 17.77 | -1.30 |
| Book Closing Line | 5,734 | 13.37 | 16.89 | +0.60 |
Win Probability predicted home-win chance vs. outcome
| Model | Games | Log Loss | Brier | Accuracy |
|---|---|---|---|---|
| Baseline (ML) | 5,531 | 0.5538 | 0.1879 | 71.0% |
| Baseline Plus (ML) | 5,425 | 0.5533 | 0.1878 | 70.6% |
| Baseline Tuned (ML) | 5,531 | 0.5545 | 0.1882 | 70.7% |
| Efficiency Plus (ML) | 5,293 | 0.5534 | 0.1879 | 71.0% |
| Massey | 5,531 | 0.5584 | 0.1887 | 70.4% |
| Bradley-Terry | 5,531 | 0.6246 | 0.2076 | 68.8% |
| Weighted Bradley-Terry | 5,531 | 0.6835 | 0.2157 | 68.7% |
| Adjusted Efficiency | 5,531 | 0.5581 | 0.1887 | 70.3% |
| Book Closing Line | 5,729 | 0.5129 | 0.1732 | 73.0% |
vs. Closing Line identical games only — the honest head-to-head
| Model | Games | Model MAE | Book MAE | Δ | ATS | O/U | CLV |
|---|---|---|---|---|---|---|---|
| Baseline (ML) | 5,487 | 9.36 | 8.89 | +0.46 | 51.0% (5,487) | 50.1% (5,513) | — |
| Baseline Plus (ML) | 5,381 | 9.28 | 8.86 | +0.42 | 50.7% (5,381) | 50.5% (5,407) | — |
| Baseline Tuned (ML) | 5,487 | 9.40 | 8.89 | +0.51 | 50.3% (5,487) | 50.7% (5,513) | — |
| Efficiency Plus (ML) | 5,249 | 9.17 | 8.87 | +0.30 | 50.3% (5,249) | 50.6% (5,275) | — |
| Massey | 5,487 | 9.53 | 8.89 | +0.64 | 50.3% (5,487) | — | — |
| Massey Totals | 0 | — | — | — | — | 49.6% (5,513) | — |
| Adjusted Efficiency | 5,487 | 9.52 | 8.89 | +0.63 | 50.2% (5,487) | 49.5% (5,513) | — |
All columns pair each model with the book closing line on the same games, so coverage differences can't skew the comparison. ATS/O-U count only games where the model disagrees with the closing number (pushes excluded) — 52.4% is breakeven at standard −110 vig. CLV counts games where the line moved off the opener and the model had taken a side against it; above 50% means the market tended to move toward the model. Not betting advice.
Month by Month do the models close the gap to the book as the season develops?
Monthly accuracy over the full season (the window selector does not apply here). Lower is better for both metrics; the dashed line is the book benchmark. Early-season months reflect ratings that are still converging on limited games.
By Conference where does the model beat the book?
| Conference | Games | Model MAE | Book MAE | Δ vs Book | Winner % |
|---|---|---|---|---|---|
| Big South Conference | 164 | 9.05 | 9.50 | -0.45 | 77.4% |
| Horizon League | 188 | 8.91 | 8.99 | -0.08 | 69.1% |
| Conference USA | 204 | 9.00 | 8.99 | +0.02 | 62.3% |
| Southland Conference | 195 | 8.34 | 8.25 | +0.09 | 70.8% |
| Southern Conference | 179 | 8.65 | 8.51 | +0.14 | 69.8% |
| Patriot League | 183 | 8.80 | 8.62 | +0.18 | 67.8% |
| Southeastern Conference | 354 | 10.18 | 9.94 | +0.24 | 76.0% |
| Atlantic Sun Conference | 221 | 9.26 | 9.00 | +0.25 | 71.5% |
| Mid-American Conference | 235 | 9.18 | 8.90 | +0.28 | 73.6% |
| Sun Belt Conference | 257 | 9.18 | 8.90 | +0.28 | 66.1% |
| Big Sky Conference | 175 | 8.87 | 8.59 | +0.29 | 70.3% |
| Missouri Valley Conference | 207 | 9.84 | 9.47 | +0.37 | 69.6% |
| Atlantic 10 Conference | 277 | 9.02 | 8.62 | +0.39 | 68.2% |
| Southwestern Athletic Conference | 222 | 9.70 | 9.29 | +0.41 | 68.5% |
| Coastal Athletic Association | 240 | 9.06 | 8.62 | +0.45 | 69.6% |
| Big East Conference | 221 | 8.99 | 8.54 | +0.45 | 72.9% |
| Mountain West Conference | 230 | 9.74 | 9.27 | +0.47 | 73.9% |
| Big 12 Conference | 343 | 9.82 | 9.35 | +0.47 | 76.4% |
| West Coast Conference | 233 | 9.63 | 9.15 | +0.47 | 76.4% |
| Big West Conference | 193 | 9.39 | 8.90 | +0.48 | 67.9% |
| Big Ten Conference | 368 | 10.02 | 9.48 | +0.54 | 75.8% |
| Summit League | 159 | 10.23 | 9.68 | +0.54 | 67.9% |
| Ivy League | 136 | 9.65 | 9.10 | +0.55 | 66.9% |
| Atlantic Coast Conference | 373 | 9.00 | 8.44 | +0.56 | 74.3% |
| American Conference | 251 | 8.69 | 8.11 | +0.57 | 72.1% |
| Northeast Conference | 179 | 8.86 | 8.23 | +0.63 | 74.9% |
| Western Athletic Conference | 125 | 8.79 | 8.16 | +0.64 | 72.0% |
| America East Conference | 167 | 9.90 | 9.24 | +0.66 | 74.3% |
| Metro Conference | 231 | 9.33 | 8.65 | +0.68 | 69.7% |
| Ohio Valley Conference | 179 | 10.20 | 9.44 | +0.76 | 66.5% |
| Mid-Eastern Athletic Conference | 149 | 9.25 | 8.10 | +1.16 | 71.8% |
Paired comparison: only games where both the selected model and the book closing line made a spread prediction, sorted best-for-the-model first. Cross-conference games count toward both teams' conferences. Small samples swing wildly — read the Games column before drawing conclusions.
Calibration when a model says 70%, does the home team win 70% of the time?
Predictions are grouped into deciles of predicted home-win probability; each point compares the decile's average prediction (x) with the actual home-win rate (y). A perfectly calibrated model follows the dashed diagonal.