The Most Accurate NFL Prediction System
Last week I posted a link to ThePredictionTracker.com, a site that tracks predictions of many NFL prediction systems. I love this site because so many people are out there making predictions, but rarely do we ever get an honest look at how accurate they've been. It even includes accuracies from previous years.
I'm especially pleased because based on the records of the other systems, the one used here at NFL Stats has been the most accurate NFL prediction system since its inception. (Please forgive the self-congratulations.) The NFL Stats logistic regression efficiency model has been estimating game outcome probabilities over the past two seasons with the highest total accuracy rate of any system.
Admittedly, there is some amount of luck involved in predicting sports outcomes, which is why various systems should be compared over more than a single season. You might notice that the system that is doing the best so far this year (Greenfield's Team Rankings system) was among the very worst in 2006. This would suggest the system is prone to a very high degree of luck. Systems with true predictive value are still susceptible to luck, but should still rank above average fairly consistently.
In 2006 the NFL Stats model accurately predicted the winning team in 122 out of 195 games for a rate of 62.6%. So far in 2007, the model has been correct in 58 out of 82 games for a rate of 70.7%. The two-year total is 180 correct out of 277 games for an accuracy rate of 65.0%. (Although 65% doesn't sound very impressive, 2006 was an unusually difficult year to predict.)
I added the two-year totals for all the other systems listed at ThePredictionTracker.com. I included systems that made predictions in both 2006 and 2007 with at least 100 total games predicted. Only 3 of the systems were excluded. In total, there were 61 systems compared. The table below lists each system and its 2-year record.
| System | Correct | Total Games | Accuracy % |
| BBNFLStats | 180 | 277 | 65.0 |
| Deslock Rankings | 160 | 249 | 64.3 |
| Pigskin Herald | 170 | 269 | 63.2 |
| ARGH Power Ratings | 212 | 337 | 62.9 |
| Hank Trexler | 249 | 397 | 62.7 |
| Least Squares | 249 | 397 | 62.7 |
| Logistic Regression | 190 | 303 | 62.7 |
| Mike Greenfield | 172 | 275 | 62.5 |
| Eric Hollobaugh | 163 | 261 | 62.5 |
| Roger Johnson | 189 | 303 | 62.4 |
| Brian Gabrielle | 247 | 397 | 62.2 |
| Sonny Moore | 247 | 397 | 62.2 |
| Beck Elo | 246 | 397 | 62.0 |
| Line (updated) | 246 | 397 | 62.0 |
| Pythagorean Ratings | 246 | 397 | 62.0 |
| Sagarin | 246 | 397 | 62.0 |
| Sagarin Elo | 245 | 397 | 61.7 |
| Stephen Kerns | 245 | 397 | 61.7 |
| System Median | 245 | 397 | 61.7 |
| Computer Adjusted Line | 245 | 397 | 61.7 |
| CPA Rankings | 244 | 397 | 61.5 |
| Least Abs. Val Reg | 244 | 397 | 61.5 |
| Steven Jens | 244 | 397 | 61.5 |
| System Average | 244 | 397 | 61.5 |
| John Coffey | 169 | 275 | 61.5 |
| CPA Retro Rankings | 243 | 397 | 61.2 |
| Lee Burdorf | 243 | 397 | 61.2 |
| Sagarin Predictive | 243 | 397 | 61.2 |
| Sports Report Elo | 233 | 381 | 61.2 |
| Viacheslav | 233 | 381 | 61.2 |
| NutShell Retro | 223 | 365 | 61.1 |
| Kambour Rating | 242 | 397 | 61.0 |
| Reistertown Index | 242 | 397 | 61.0 |
| Line (opening) | 241 | 397 | 60.7 |
| Scoring Effeciency | 240 | 396 | 60.6 |
| Pigskin Index | 240 | 397 | 60.5 |
| PerformanZ Ratings | 239 | 396 | 60.4 |
| Jeff Imes | 237 | 393 | 60.3 |
| Matthews Grid | 161 | 267 | 60.3 |
| Jeff Self | 168 | 279 | 60.2 |
| Covers.com | 229 | 381 | 60.1 |
| Kasulis Enhanced Spread | 238 | 397 | 60.0 |
| Dokter Entropy | 228 | 381 | 59.8 |
| The Sports Report | 228 | 381 | 59.8 |
| What If Sports | 171 | 286 | 59.8 |
| JFM Power Ratings | 237 | 397 | 59.7 |
| Stat Fox | 237 | 397 | 59.7 |
| Computer Adjusted Li | 159 | 267 | 59.6 |
| Bihl Rankings | 147 | 247 | 59.5 |
| Ed Bemiss | 155 | 263 | 58.9 |
| Ashby AccuRatings | 233 | 397 | 58.7 |
| Dunkel Index | 233 | 397 | 58.7 |
| Sport Trends | 233 | 397 | 58.7 |
| Geoff Rout | 122 | 210 | 58.1 |
| Tom Benson | 230 | 397 | 57.9 |
| LS - w/ team HFA | 228 | 397 | 57.4 |
| Kasulis Enhanced Spr | 153 | 267 | 57.3 |
| Grid Iron Gold | 192 | 340 | 56.5 |
| NutShell Sports | 220 | 397 | 55.4 |
| TSR Slots | 206 | 381 | 54.1 |
| Sports Report Pred | 68 | 130 | 52.3 |
(One caveat: my model is based solely on current season data so no public predictions were made prior to week 4. Many of the other systems make predictions beginning at week 1. However, my model would have been 33/47-- 70.2% correct for weeks 1-3 in 2007, which would have actually improved the model's record. These weeks were not published and therefore I did not include them in the comparison above. Note: Week 1 was predicted using just home teams, and one game in week 3 was predicted 50/50 and was excluded. )
2 comments:
Why don't you just use linear regression or something to predict the final score margins. You don't even have to publish it on your site, just send it to PredictionTracker, that way you can at least get your SU predictions listed.
That's true. I do have a linear point model I used last year that's on the shelf ready to go. One thing I've learned is that predicting against the spread is pretty futile. If you look at the PredictionTracker site, you'll notice that none of the models do well consistently against the spread. With 60+ different models, there is bound to be several each year that do fairly well just due to luck, and no one model appears to do well from year to year.
I do have some ideas about how to approach point differentials other than a direct linear regression.
I'd like to further develop my interaction model that considers match-ups between opposing squads, i.e. a great pass offense vs weak pass defense.
Another idea is to predict scoring drives and not points, then factor in TD to FG ratios.
I think the idea with the most potential is to develop a binary model that predicts beat spread/doesn't beat spread, and doesn't attempt to estimate the actual point difference.
My recommendation for those that actually bet on games is to select the games in which a reliable model predicts will be a SU upset. You'll not only probably win SU, but you also get the benefit of the points.
Post a Comment